BABAR-PUB-04/030 
SLAC-PUB-10808 
hep-ex 0411016 
Submitted to Physical Review D 
November, 2004 



Ambiguity- Free Measurement of cos 2/3: Time-Integrated and 
Time-Dependent Angular Analyses of B — > J/ift Kir 



The BABAR Collaboration 



B. Aubert, R. Barate, D. Boutigny, F. Couderc, J.-M. Gaillard, 
Y. Karyotakis, J. P. Lees, V. Poireau, V. Tisserand, and A. Zghiche 
Laboratoire de Physique des Particules, F-74941 Annecy-le-Vieux, France 

A. Palano and A. Pompili 

Universita di Bari, Dipartimento di Fisica and INFN, 1-70126 Bari, Italy 

J. C. Chen, N. D. Qi, G. Rong, P. Wang, and Y. S. Zhu 
Institute of High Energy Physics, Beijing 100039, China 

G. Eigen, I. Ofte, and B. Stugu 
University of Bergen, Inst, of Physics, N-5007 Bergen, Norway 

G. S. Abrams, A. W. Borgland, A. B. Breon, D. N. Brown, J. Button-Shafer, R. N. Cahn, 
E. Charles, C. T. Day, M. S. Gill, A. V. Gritsan, Y. Groysman, R. G. Jacobsen, R. W. Kadel, 
J. Kadyk, L. T. Kerth, Yu. G. Kolomensky, G. Kukartsev, G. Lynch, L. M. Mir, P. J. Oddone, 
T. J. Orimoto, M. Pripstein, N. A. Roe, M. T. Ronan, V. G. Shelkov, and W. A. Wenzel 
Lawrence Berkeley National Laboratory and University of California, Berkeley, CA 94720, USA 

M. Barrett, K. E. Ford, T. J. Harrison, A. J. Hart, C. M. Hawkes, S. E. Morgan, and A. T. Watson 
University of Birmingham, Birmingham, B15 2TT, United Kingdom 

M. Fritsch, K. Goetzen, T. Held, H. Koch, B. Lewandowski, M. Pelizaeus, and M. Steinke 
Ruhr Universitdt Bochum, Institut fur Experimentalphysik 1, D-44^80 Bochum, Germany 



J. T. Boyd, N. Chevalier, W. N. Cottingham, M. P. Kelly, T. E. Latham, and F. F. Wilson 
University of Bristol, Bristol BS8 1TL, United Kingdom 



T. Cuhadar-Donszelmann, C. Hearty, N. S. Knecht, T. S. Mattison, J. A. McKenna, and D. Thiessen 
University of British Columbia, Vancouver, BC, Canada V6T 1Z1 



A. Khan, P. Kyberd, and L. Teodorescu 
Brunei University, Uxbridge, Middlesex UB8 3PH, United Kingdom 



A. E. Blinov, V. E. Blinov, V. P. Druzhinin, V. B. Golubev, V. N. Ivanchenko, E. A. Kravchcnko, 
A. P. Onuchin, S. I. Serednyakov, Yu. I. Skovpen, E. P. Solodov, and A. N. Yushkov 
Budker Institute of Nuclear Physics, Novosibirsk 630090, Russia 



D. Best, M. Bruinsma, M. Chao, I. Eschrich, D. Kirkby, A. J. Lankford, 
M. Mandelkern, R. K. Mommsen, W. Roethel, and D. P. Stoker 
University of California at Irvine, Irvine, CA 92697, USA 



2 



C. Buchanan and B. L. Hartfiel 

University of California at Los Angeles, Los Angeles, CA 90024, USA 

S. D. Foulkes, J. W. Gary, B. C. Shcn, and K. Wang 
University of California at Riverside, Riverside, CA 92521, USA 

D. del Re, H. K. Hadavand, E. J. Hill, D. B. MacFarlane, H. P. Paar, Sh. Rahatlou, and V. Sharma 

University of California at San Diego, La Jolla, CA 92093, USA 

J. Adam Cunha, J. W. Berryhill, C. Campagnari, B. Dahmcs, T. M. Hong, 
O. Long, A. Lu, M. A. Mazur, J. D. Richman, and W. Verkerke 
University of California at Santa Barbara, Santa Barbara, CA 93106, USA 

T. W. Beck, A. M. Eisner, C. A. Heusch, J. Kroseberg, W. S. Lockman, G. Nesom, 
T. Schalk, B. A. Schumm, A. Scidcn, P. Spradlin, D. C. Williams, and M. G. Wilson 

University of California at Santa Cruz, Institute for Particle Physics, Santa Cruz, CA 95064, USA 

J. Albert, E. Chen, G. P. Dubois-Felsmann, A. Dvoretskii, D. G. Hitlin, 
I. Narsky, T. Piatenko, F. C. Porter, A. Ryd, A. Samuel, and S. Yang 
California Institute of Technology, Pasadena, CA 91125, USA 

S. Jayatilleke, G. Mancinelli, B. T. Meadows, and M. D. Sokoloff 

University of Cincinnati, Cincinnati, OH 45221, USA 

F. Blanc, P. Bloom, S. Chen, W. T. Ford, U. Nauenberg, A. Olivas, P. Rankin, J. G. Smith, J. Zhang, and L. Zhang 

University of Colorado, Boulder, CO 80309, USA 

A. Chen, J. L. Harton, A. Soffer, W. H. Toki, R. J. Wilson, and Q. Zeng 
Colorado State University, Fort Collins, CO 80523, USA 

D. Altenburg, T. Brandt, J. Brose, M. Dickopp, E. Feltresi, A. Hauke, H. M. Lacker, R. Muller-Pfefferkorn, 
R. Nogowski, S. Otto, A. Petzold, J. Schubert, K. R. Schubert, R. Schwierz, B. Spaan, and J. E. Sundermann 

Technische Universitat Dresden, Institut fur Kern- und Teilchenphysik, D-01062 Dresden, Germany 

D. Bernard, G. R. Bonneaud, F. Brochard, P. Grenier, S. Schrenk, Ch. Thicbaux, G. Vasilciadis, and M. Verderi 

Ecole Polytechnique, LLR, F-91128 Palaiseau, France 

D. J. Bard, P. J. Clark, D. Lavin, F. Muheim, S. Playfer, and Y. Xic 
University of Edinburgh, Edinburgh EH9 3JZ, United Kingdom 

M. Andrcotti, V. Azzolini, D. Bettoni, C. Bozzi, R. Calabrese, 
G. Cibinetto, E. Luppi, M. Negrini, L. Piemontese, and A. Sarti 
Universita di Ferrara, Dipartimento di Fisica and INFN, 1-44100 Ferrara, Italy 

E. Treadwell 

Florida A&M University, Tallahassee, FL 32307, USA 

F. Anulli, R. Baldini-Ferroli, A. Calcaterra, R. de Sangro, 
G. Finocchiaro, P. Pattcri, I. M. Pcruzzi, M. Piccolo, and A. Zallo 
Laboratori Nazionali di Frascati dell'INFN, 1-00044 Frascati, Italy 

A. Buzzo, R. Capra, R. Contri, G. Crosetti, M. Lo Vctere, M. Macri, 
M. R. Monge, S. Passaggio, C. Patrignani, E. Robutti, A. Santroni, and S. Tosi 
Universita di Genova, Dipartimento di Fisica and INFN, 1-16146 Genova, Italy 

S. Bailey, G. Brandenburg, K. S. Chaisanguanthum, M. Morii, and E. Won 

Harvard University, Cambridge, MA 02138, USA 



3 

R. S. Dubitzky, U. Langenegger, J. Marks, and U. Uwer 
Universitdt Heidelberg, Physikalisches Institut, Philosophenweg 12, D-69120 Heidelberg, Germany 

W. Bhimji, D. A. Bowerman, P. D. Dauncey, U. Egedc, J. R. Gaillard, 
G. W. Morton, J. A. Nash, M. B. Nikolich, and G. P. Taylor 
Imperial College London, London, SW7 2AZ, United Kingdom 

M. J. Charles, G. J. Grcnier, and U. Mallik 

University of Iowa, Iowa City, IA 52242, USA 

J. Cochran, H. B. Crawley, J. Lamsa, W. T. Meyer, S. Prcll, E. I. Rosenberg, A. E. Rubin, and J. Yi 

Iowa State University, Ames, IA 50011-3160, USA 

M. Biasini, R. Covarelli, and M. Pioppi 

Universita di Perugia, Dipartimento di Fisica and INFN, 1-06100 Perugia, Italy 

M. Davier, X. Giroux, G. Grosdidier, A. Hocker, S. Laplace, F. Le Diberder, V. Lepeltier, 
A. M. Lutz, T. C. Petersen, S. Plaszczynski, M. H. Schune, L. Tantot, and G. Wormser 
Laboratoire de I'Accelerateur Lineaire, F-91898 Orsay, France 

C. H. Cheng, D. J. Lange, M. C. Simani, and D. M. Wright 

Lawrence Livermore National Laboratory, Livermore, CA 94550, USA 

A. J. Bcvan, C. A. Chavez, J. P. Coleman, I. J. Forstcr, J. R. Fry, E. Gabathulcr, 
R. Garnet, D. E. Hutchcroft, R. J. Parry, D. J. Payne, R. J. Sloane, and C. Touramanis 
University of Liverpool, Liverpool L69 72E, United Kingdom 

C. M. Cormack and F. Di Lodovico 

Queen Mary, University of London, El 4NS, United Kingdom 

C. L. Brown, G. Cowan, R. L. Flack, H. U. Flaecher, M. G. Green, 
P. S. Jackson, T. R. McMahon, S. Ricciardi, F. Salvatore, and M. A. Winter 
University of London, Royal Holloway and Bedford New College, Egham, Surrey TW20 OEX, United Kingdom 

D. Brown and C. L. Davis 
University of Louisville, Louisville, KY 40292, USA 

J. Allison, N. R. Barlow, R. J. Barlow, M. C. Hodgkinson, G. D. Lafferty, A. J. Lyon, and J. C. Williams 

University of Manchester, Manchester M13 9PL, United Kingdom 

A. Farbin, W. D. Hulsbergen, A. Jawahery, D. Kovalskyi, C. K. Lae, V. Lillard, and D. A. Roberts 
University of Maryland, College Park, MB 20142, USA 

G. Blaylock, C. Dallapiccola, S. S. Hertzbach, R. Kofler, 
V. B. Koptchev, T. B. Moore, S. Saremi, H. Staengle, and S. Willocq 

University of Massachusetts, Amherst, MA 01003, USA 

R. Cowan, G. Sciolla, S. J. Sekula, F. Taylor, and R. K. Yamamoto 
Massachusetts Institute of Technology, Laboratory for Nuclear Science, Cambridge, MA 02139, USA 

D. J. J. Mangeol, P. M. Patel, and S. H. Robertson 
McGill University, Montreal, QC, Canada H3A 2T8 

A. Lazzaro, V. Lombardo, and F. Palombo 

Universita di Milano, Bipartimento di Fisica and INFN, 1-20133 Milano, Italy 



J. M. Bauer, L. Cremaldi, V. Eschenburg, R. Godang, R. Kroeger, 
J. Reidy, D. A. Sanders, D. J. Summers, and H. W. Zhao 



4 

University of Mississippi, University, MS 38677, USA 

S. Brunct, D. Cote, and P. Taras 
Universite de Montreal, Laboratoire Rene J. A. Levesque, Montreal, QC, Canada H3C 337 

H. Nicholson 

Mount Holyoke College, South Hadley, MA 01075, USA 

N. Cavallo,* F. Fabozzi,* C. Gatto, L. Lista, D. Monorchio, P. Paolucci, D. Piccolo, and C. Sciacca 
Universita di Napoli Federico II, Dipartimento di Scienze Fisiche and INFN, 1-80126, Napoli, Italy 

M. Baak, H. Bulten, G. Raven, H. L. Snock, and L. Wildcn 

NIKHEF, National Institute for Nuclear Physics and High Energy Physics, NL-1009 DB Amsterdam, The Netherlands 

C. P. Jessop and J. M. LoSecco 
University of Notre Dame, Notre Dame, IN 46556, USA 

T. Allmcndingcr, K. K. Gan, K. Honscheid, D. Hufnagel, H. Kagan, 
R. Kass, T. Pulliam, A. M. Rahimi, R. Ter-Antonyan, and Q. K. Wong 
Ohio State University, Columbus, OH 43210, USA 

J. Brau, R. Frcy, O. Igonkina, C. T. Potter, N. B. Sinev, D. Strom, and E. Torrence 
University of Oregon, Eugene, OR 97403, USA 

F. Colecchia, A. Dorigo, F. Galeazzi, M. Margoni, M. Morandin, 

M. Posocco, M. Rotondo, F. Simonetto, R. Stroili, G. Tiozzo, and C. Voci 
Universita di Padova, Dipartimento di Fisica and INFN, 1-35131 Padova, Italy 

M. Benayoun, H. Briand, J. Chauveau, P. David, Ch. de la Vaissiere, L. Del Buono, O. Hamon, 

M. J. J. John, Ph. Leruste, J. Malcles, J. Ocariz, M. Pivk, L. Roos, S. T'Jampens, and G. Therin 

Universites Paris VI et VII, Laboratoire de Physique Nucleaire et de Hautes Energies, F-75252 Paris, France 

P. F. Manfredi and V. Re 
Universita di Pavia, Dipartimento di Elettronica and INFN, 1-27100 Pavia, Italy 

P. K. Behera, L. Gladncy, Q. H. Guo, and J. Panetta 
University of Pennsylvania, Philadelphia, PA 19104, USA 

C. Angelini, G. Batignani, S. Bcttarini, M. Bondioli, F. Bucci, G. Calderini, 
M. Carpinelli, F. Forti, M. A. Giorgi, A. Lusiani, G. Marchiori, F. Martinez- Vidal,^ 
M. Morganti, N. Neri, E. Paoloni, M. Rama, G. Rizzo, F. Sandrelli, and J. Walsh 

Universita di Pisa, Dipartimento di Fisica, Scuola Normale Superiore and INFN, 1-56127 Pisa, Italy 

M. Haire, D. Judd, K. Paick, and D. E. Wagoner 
Prairie View A&M University, Prairie View, TX 77446, USA 

N. Danielson, P. Elmer, Y. P. Lau, C. Lu, V. Miftakov, J. Olsen, A. J. S. Smith, and A. V. Tclnov 

Princeton University, Princeton, NJ 08544, USA 

F. Bellini, R. Faccini, F. Ferrarotto, F. Ferroni, M. Gaspero, L. Li Gioi, 

M. A. Mazzoni, S. Morganti, M. Pierini, G. Piredda, F. Safai Tehrani, and C. Voena 
Universita di Roma La Sapienza, Dipartimento di Fisica and INFN, 1-00185 Roma, Italy 

G. Cavoto 

Princeton University, Princeton, NJ 08544, USA and 
Universita di Roma La Sapienza, Dipartimento di Fisica and INFN, 1-00185 Roma, Italy 

S. Christ, G. Wagner, and R. Waldi 



5 

Universitat Rostock, D- 18 051 Rostock, Germany 

T. Adye, N. De Groot, B. Franck, N. I. Geddcs, G. P. Gopal, and E. O. Olaiya 
Rutherford Appleton Laboratory, Chilton, Didcot, Oxon, 0X11 OQX, United Kingdom 

R. Aleksan, S. Emery, A. Gaidot, S. F. Ganzhur, P.-F. Giraud, G. Hamel de Monchenault, 
W. Kozanecki, M. Legendre, G. W. London, B. Mayer, G. Schott, G. Vasseur, Ch. Yeche, and M. Zito 
DSM/Dapnia, CEA/Saclay, F-91191 Gif-sur-Yvette, France 

M. V. Purohit, A. W. Weidemann, J. R. Wilson, and F. X. Yumiceva 
University of South Carolina, Columbia, SC 29208, USA 

T. Abe, D. Aston, R. Bartoldus, N. Berger, A. M. Boyarski, O. L. Buchmucllcr, R. Claus, 
M. R. Convery, M. Cristinziani, G. De Nardo, D. Dong, J. Dorfan, D. Dujmic, W. Dunwoodie, 
E. E. Elscn, S. Fan, R. C. Field, T. Glanzman, S. J. Gowdy, T. Hadig, V. Halyo, C. Hast, T. Hryn'ova, 
W. R. Inncs, M. H. Kelsey, P. Kim, M. L. Kocian, D. W. G. S. Lcith, J. Libby, S. Luitz, V. Luth, 
H. L. Lynch, H. Marsiske, R. Messner, D. R. Muller, C. P. O'Grady, V. E. Ozcan, A. Pcrazzo, 
M. Perl, S. Petrak, B. N. Ratcliff, A. Roodman, A. A. Salnikov, R. H. Schindler, J. Schwiening, 
G. Simi, A. Snyder, A. Soha, J. Stelzer, D. Su, M. K. Sullivan, J. Va'vra, S. R. Wagner, M. Weaver, 
A. J. R. Weinstein, W. J. Wisniewski, M. Wittgen, D. H. Wright, A. K. Yarritu, and C. C. Young 
Stanford Linear Accelerator Center, Stanford, CA 94309, USA 

P. R. Burchat, A. J. Edwards, T. I. Meyer, B. A. Petersen, and C. Roat 
Stanford University, Stanford, CA 94305-4060, USA 

M. Ahmed, S. Ahmed, M. S. Alam, J. A. Ernst, M. A. Saeed, M. Saleem, and F. R. Wapplcr 
State University of New York, Albany, NY 12222, USA 

W. Bugg, M. Krishnamurthy, and S. M. Spanier 

University of Tennessee, Knoxville, TN 37996, USA 

R. Eckmann, H. Kim, J. L. Ritchie, A. Satpathy, and R. F. Schwitters 

University of Texas at Austin, Austin, TX 78712, USA 

J. M. Izen, I. Kitayama, X. C. Lou, and S. Ye 

University of Texas at Dallas, Richardson, TX 75083, USA 

F. Bianchi, M. Bona, F. Gallo, and D. Gamba 
Universitd di Torino, Dipartimento di Fisica Sperimentale and LNFN, 1-10125 Torino, Italy 

L. Bosisio, C. Cartaro, F. Cossutti, G. Delia Ricca, S. Dittongo, 
S. Grancagnolo, L. Lanccri, P. Poropat,^ L. Vitalc, and G. Vuagnin 

Universitd di Trieste, Dipartimento di Fisica and INFN, 1-34127 Trieste, Italy 

R. S. Panvini 

Vanderbilt University, Nashville, TN 37235, USA 

Sw. Banerjee, C. M. Brown, D. Fortin, P. D. Jackson, R. Kowalewski, J. M. Roney, and R. J. Sobie 

University of Victoria, Victoria, BC, Canada V8W 3P6 

J. J. Back, P. F. Harrison, and G. B. Mohanty 
Department of Physics, University of Warwick, Coventry CV4 7AL, United Kingdom 

H. R. Band, X. Chen, B. Cheng, S. Dasu, M. Datta, A. M. Eichcnbaum, K. T. Flood, 
M. Graham, J. J. Hollar, J. R. Johnson, P. E. Kutter, H. Li, R. Liu, A. Mihalyi, Y. Pan, 
R. Prepost, P. Tan, J. H. von Wimmersperg-Toeller, J. Wu, S. L. Wu, and Z. Yu 

University of Wisconsin, Madison, WI 53706, USA 



6 



M. G. Greene and H. Neal 
Yale University, New Haven, CT 06511, USA 
(Dated: February 7, 2008) 

We present results on B — » J/ip Kir decays using e + e~ annihilation data collected with the BABAR 
detector at the T(4S) resonance. The detector is located at the PEP-II asymmetric-energy storage 
ring facility at the Stanford Linear Accelerator Center. Using approximately 88 million BB pairs, 
we measure the decay amplitudes for the flavor eigenmodes and observe strong-phase differences 
indicative of final-state interactions with a significance of 7.6 standard deviations. We use the 
interference between the Kir S-wave and P-wave amplitudes in the region of the if* (892) to resolve 
the ambiguity in the determination of these strong phases. We then perform an ambiguity-frcc 
measurement of cos 2/3 using the angular and time-dependent asymmetry in B — * J/tp K*° (Kgir ) 
decays. With sin2/3 fixed at its measured value and cos2/3 treated as an independent parameter, we 
find cos 2/3 = 2.72±g;!$(stat) ± 0.27(syst), determining the sign of cos 2/3 to be positive at 86% CL. 

PACS numbers: 13.25.Hw, 12.15.Hh, 14.40.Nd, 11.30.Er 



The Standard Model of electroweak interactions de- 
scribes CP violation in weak interactions of quarks by 
the presence of a nonzero phase in the three-generation 
Cabibbo-Kobayashi-Maskawa (CKM) quark-mixing ma- 
trix [1]. In this framework, the CP-violation parameter 
sin2/3 can be measured by examining the proper-time dis- 
tributions of neutral B-meson decays to final states con- 
taining a charmonium meson and a neutral kaon. The 
Belle [2] and BABAR [3] experiments have recently per- 
formed precise measurements of sin 2/3, leading to a world 
average of 0.731 ± 0.056 [4]. These measurements de- 
termine (3 up to a four-fold ambiguity, corresponding to 
the two different signs of cos2/3 and the transformation 

(3 -> 7T + (3. 

One of the possible values of (3 is compatible with mea- 
surements of other quantities that constrain the Unitarity 
Triangle [4]. However, it is still possible that, because of 
contributions from new physics, the actual value of (3 is 
one of the three other values consistent with the mea- 
surement of sin 2(3 [5-7] . A measurement of the sign of 
cos2/3 would cither agree with the standard interpreta- 
tion j3 « 0.41 and with its indistinguishable nonstandard 
alternative (3 ~ 0.41 + w, or would exclude these and 
instead imply the nonstandard solutions (3 ~ 1.16 and 

(3 W 1.16 + 7T. 

Several strategies to determine cos 2/3 have been pro- 
posed [6], [8-13]. In particular, cos 2/3 appears as a factor 
in the interference between the CP-odd and the two CP- 
even amplitudes in the time- and angle-dependent distri- 
bution describing the decay B -» J/ipK*° {K*° -» K° s tt , 
J/ip — » £ + £~) [14-17]. However, neither this distribu- 
tion nor the time-integrated angular distributions of the 
companion channels P° — > J/ipK*°(K*° — > K + ir~) and 
B+ — > J/ipK* + (and related charge-conjugate decays) 
can resolve a two-fold ambiguity in the relative strong 
phases among the three B — > J/ipK* decay amplitudes. 



*Also with Universita della Basilicata, Potenza, Italy 

tAlso with IFIC, Institute dc Fi'sica Corpuscular, CSIC- 

Universidad de Valencia, Valencia, Spain 

t Deceased 



This leaves an overall sign ambiguity in cos 2/3 [18, 19]. 
Resolving the ambiguity from these partial waves alone 
would require the measurement of the polarization of 
the leptons from the J/ip decay [20]. This could be 
done in principle using J/ip — ► ^+ decays or with 
ip(2S) — ► t+t^ decays by measuring the lepton polariza- 
tions. Such measurements are not feasible today. The- 
oretical arguments, based on the analysis of s-quark he- 
licity conservation, suggest a "preferred" set of strong 
phases [21], but cannot guarantee the validity of this set. 

In this analysis, we use the known dependence on Kir 
invariant mass of the relative phase between the S-w&ve 
and P-wave Kir I = 1/2 scattering amplitudes in the 
vicinity of the if* (892) to resolve the two- fold ambiguity 
in the relative strong phases among the three amplitudes 
for B — > J/ipK* . The dominant P-wave has the canon- 
ical Breit-Wigner form with a phase 5p that increases 
rapidly with Kir invariant mass vcik-k-, while the 5-wave 
phase 6s increases slowly with itik-w [22]. Accordingly, 
6s ~ bp, where 6p is assumed to be common to the three 
B — ► J/ip Kir P-wave amplitudes, is expected to decrease 
rapidly as tukit increases from below to above the K* 
resonance. We find that one solution for 5 S — 5 P yields 
this expected behaviour while the other has precisely the 
opposite behavior. In this way, the ambiguity is resolved, 
so that the subsequent time-dependent analysis yields a 
value of cos2/3 that is free of the associated sign ambigu- 
ity. 

We perform an angular analysis of the B — > J/ip K* de- 
cay mode and measure cos2/3 on a sample of (88±1) x 10 6 
BB pairs collected with the BABAR detector at the PEP- 
II asymmetric-energy B Factory. These data correspond 
to an integrated luminosity of 81.9 fb _1 recorded at the 
T(45) resonance. The analysis is performed in three dis- 
tinct stages. 

In the first stage the time-integrated angular dis- 
tributions describing the decay channels B° — ► 
J/ipK*°,K*° -> K+tt- and B+ -» J/iPK*+,K*+ -> 
iY°7r+ and K*+ -> K+ir°, together with those of the 
related charge-conjugate modes, are analyzed assuming 
that the Kir system may be described entirely in terms 
of P-wave amplitudes. The weak process b — > ccs is a 



7 



TABLE I: The B — ► J/tpK* (892) amplitude parameters (described in Sec. I) as measured by several experiments. The results 
in Ref. [19] are superseded by this work. Note that the phases are subject to a two- fold ambiguity, as described by Eq. (4). 





\M 2 


l^xl 2 


$11 — $o ( rad) 


$x — $o ( rad) 


CLEO [23] 


0.52 ±0.07 ±0.04 


0.16 ±0.08 ±0.04 


3.00 ±0.37 ±0.04 


-0.11 ±0.46 ±0.03 


CDF [24] 


0.59 ±0.06 ±0.01 


0.13to£jj ±0.06 


2.2 ±0.5 ±0.1 


-0.6 ±0.5 ±0.1 


BABAR [19] 


0.597 ±0.028 ±0.024 


0.160 ± 0.032 ±0.014 


2.50 ±0.20 ±0.08 


-0.17 ±0.16 ±0.07 


Belle [25] 


0.617 ±0.020 ±0.027 


0.192 ± 0.023 ±0.026 


2.83 ±0.19 ±0.08 


-0.09 ±0.13 ±0.06 



AI = interaction, so the amplitudes for B° and B + 
decay should be equal, as should be those for B° and 
B~ . A convenient description of the decays is provided 
in the transversity basis [14] since the related ampli- 
tudes have well-defined CP parities, which is of particu- 
lar relevance for the cos2/3 measurement. The formalism 
is described in Sec. I A and the results of its applica- 
tion to the data are presented in Sec. IV in the form of 
transversity-amplitude magnitudes and relative phases. 
There is an intrinsic mathematical ambiguity associated 
with the phases; the relevant transformation expressed in 
Eq. (4) below leaves the angular distribution unchanged. 

This ambiguity can be resolved by extending the for- 
malism to include a Ktt S-wave amplitude and then mea- 
suring the Ktt mass-dependence of its phase difference 
with respect to the P-waves. We will show that including 
a Ktt S-wave with a significant S-P interference is re- 
quired to describe the data (see Sec. V A). The extended 
angular distribution is presented in Sec. IB, and its use 
in resolving the phase ambiguity is described in Sec. V. 
This is the second stage in the analysis procedure. 

The P-wave parameters extracted in Sec. IV are only 
slightly affected by the presence of an S-wave amplitude 
since in performing the analysis the data are integrated 
over a broad (±100 MeV/c 2 ) Ktt mass interval centered 
on the if* (892). The S-P interference contributions es- 
sentially average out over this region, and since the S- 
wave intensity proves to be only a few percent of that of 
the P-wave, the presence of the S-wave can be accounted 
for by a small additional source of systematic uncertainty 
(Table V, line 7). 

The third stage of the analysis is the application of the 
time-dependent formalism to the B° — ► J/ip K*° (K®tt°) 
decay channel, as described in Sec. IC. There, the Ktt 
S-wave is omitted and the P-wave parameters are fixed 
to those obtained during the first stage of the analy- 
sis. The phase ambiguity discussed in Sec. I A translates 
into a sign ambiguity for cos2/3. The Ktt S-P inter- 
ference analysis of Sec. IV resolves the ambiguity and 
results in the assignment of a unique sign to the term 
in the time-dependent angular distribution that is pro- 
portional to cos2/3. The time-dependent analysis of the 
B° — > J/ip K*°(K®tt°) data sample, which is statistically 
independent of that used for the measurement of the am- 
plitudes, is presented in Sec. VI. We summarize the re- 
sults of the paper in Sec. VII. 

Several experiments have determined the decay am- 
plitudes in B — > J/ipK* decay. These results are sum- 




FIG. 1: Definition of the transversity angles (6k* ,6t r ,4>tr) 
and coordinate system (xtr, Vtr, Ztr)- The direction opposite 
to the B meson momentum in the J/ip rest frame is Xtr', Vtr 
is perpendicular to x tr in the plane that contains x tr and pn, 
chosen so px ytr > 0; ztr = xtr x ytr- The helicity angle 6k* 
of the K* decay is the angle between the direction opposite 
to the B meson flight direction and the kaon momentum, in 
the K* rest frame. Finally, 6 tr and <j) tr are the polar and 
azimuthal angle of the positive lepton defined in the J/ip rest 
frame. 

marized in Table I. The measurements presented here 
supersede previous BABAR results [19], which are based 
on subsets of the data used for this analysis. The data 
reconstruction and Monte Carlo simulation procedures 
have undergone significant improvement since our pre- 
vious analysis; the reconstruction of J/ipK* channels in- 
volving a 7T° has been improved (Sec. Ill) leading to a bet- 
ter purity; a dedicated background subtraction method 
has been developed (Sec. IV B and Appendix A). 

I. ANGULAR- AND TIME-DEPENDENT 
DIFFERENTIAL DECAY RATES 

The B decay amplitudes are measured from the time- 
integrated differential decay distribution, expressed in 
the transversity basis. The definitions of the transversity 
frame and the related transversity angles (9k* ,9tr,<Ptr) 
are shown in Fig. 1. 

A. The Time-Integrated J/ip K* Angular 
Distribution 

We first consider only the X*(892) Ktt mass region. 
The amplitude for longitudinal polarization of the two 
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vector mesons is Aq. There are two amplitudes for po- 
larizations of the vector mesons transverse to the decay 
axis: A\\ for parallel polarization of the two vector mesons 
and A± for their perpendicular polarization. The three 
independent amplitudes are assumed to have a common 
dependence on vhk-k- This common dependence is irrel- 
evant to the angular distribution and each of the three 
amplitudes is thus represented by a complex constant. 

In terms of the angular variables <jJ = (Ok* , Otn 4>tr), 
the time-integrated differential decay rate for the 
decay of the B meson to the J/ip (K + n~) p wavc , 



g(u;A) = i 



d 3 r 



Plr 



r d cos Ok* d cos 9 tr c 
= / 1 M|,4 | 2 + / 2 (u;)|A||| 2 + / 3 (u;)|A ± | 
/ 4 (w)9fm(AJ[A ± ) + / 6 (w)Re(A||^) H 
f 6 (u;)%m(A ± A* ), 



(1) 



where the functions /i(w) are 



/2(W 
/3(W 

/ 6 (« 



^-2 cos 2 9 K * [l - sin 2 6> tr cos 2 (f> tr ] 



— sm 2 9 K * [1 



32tt 

9 . 2 
^r- sm 
32tt 

9 . 2 
— — — sm 

32tt 

_9 1_ 

" 32^71 
_9 1_ 

32^71 



sin 2 # tr sin 2 i 



i K * sm tr , 
: 9k* sin2# tr sin0i r , 
sin 29k* sin 2 tr sin20 tr , 
sin 26 k* sin 2# tr cos </>t r . 



(2) 



Equations (1,2) have been obtained by summing over the 
unobserved lepton polarizations [20, 26, 27]. 

The symbol A denotes the transversity amplitudes for 
the decay of the B meson: A = (A ,A»,A±). We set 
\A \ 2 + L4|,| 2 + \A ± \ 2 = 1, so that g(u;A) (Eq. (l))js 
a probability density function (PDF). Wc denote by A 
the amplitudes for the B meson decay. In the absence 
of direct CP violation, we can choose a phase convention 
so that these amplitudes are related by Ao = +Aq, An = 
+An, A± = —A±, so that A± is CP-odd and Aq and An 



are CP-even. Fixing this phase convention also fixes the 
phase of the amplitude for B°-B° mixing. 

The phases Si of the amplitudes, where i — 0, ||, _L, are 
defined by Ai — \Ai\e tSi . Obviously, only differences of 
phases appear in the differential decay rate through the 
observables 



Sm(A|A|_) 
MA\\A* ) 
Zm(A ± A*) 



|A||||,4 L |sin(£ L 
|A||||A | cos((5|| - 
|^4_l||-4o| sin((5_L 



So), 
-So), 



(3) 



so that the differential decay rate (Eq.(l)) is invariant 
under the tranformation 

(<5|| - S , S ± - S ) < — ► {S - Su, it + S - 5 ± ). (4) 



This is the above-mentioned ambiguity. 

The three terms that would allow to resolve the am- 
biguity (9te(A^A±),$lm(A\\AZ) and 9le(A_ L A ')) vanish 
after summation over the unobserved final lepton polar- 
izations. 

We ensure \A \ 2 + \A^\ 2 + \Aj_\ 2 = 1 by parametrizing 
the magnitudes of the three P-decay amplitudes by 



cos 9 a 
sin 9a cos 4>a 
sin 9a sin <pA 



14)1 , 

141, 
I4i|. 



(5) 



with < 9 A < tt/2, < <p A < 7r/2. 



B. Angular Distributions Including a K it S-Wave 
Contribution 

The Kir system originating from B — ► J/tp(Kir) can, 
in principle, have any integer spin. The experiment with 
the largest Kir sample, LASS [22], showed however that 
below 1.3 GeV/c 2 , the S and P waves dominate. We have 
previously observed a broad structure [19] in the 1.1-1.3 
GeV/c 2 range of the mif, spectrum and found it to be 
compatible with a significant S'-wave amplitude contri- 
bution. When a Kir S wave in the B decay amplitude is 
included in addition to the Kir P wavc, the differential 
decay rate (Eq. (1)) becomes [27] 



Gs+p(u,m K ir;A,Ap,As) = 



i d 4 r 

r dmif ,rd cos 9k* d cos 9 tr d(ptr 



oc pq x 



A 2 P g(LJ:A) + \As\ 2 f 7 (u) + 



A P [/ 8 (w)Re (A l{ A* s ) + f 9 (w)$tm (A ± A* S ) + / 10 (o»)Re (A A* S )] 



(6) 



9 



where we have kept the notation Ok * for the (Kir) helicity 
angle; p is the K 7r-system momentum in the B rest frame 
and q is the kaon momentum in the Kir rest frame; we 
chose Ap to a be real and positive function of m^. Its 
square is indicative of the overall strength of the P-wave 
amplitudes. We represent the m^-dependent S-wave 
amplitude as As — \As\e lSs . The phases of the P-wave 
amplitudes reside in A 0} A\\, and A±. 

Using the same phase convention as for the P-wave 
amplitudes, A s = A s . The angular functions /7...10 are 



/ 9 («) = 

/io(w) = 



3 

327 
3 

~327 
3 

327 
3 

327 



2 [l — sin 2 Otr cos 2 tr ] , 
V6 sin sin 2 6>i r sin 2tfi tr , 
V6 sin Ok* sin 2# tr cos tr , 
4-\/3 cos [l — sin 2 cos 2 tr ] 



(7) 



At a given turk, the normalization is obtained by in- 
troducing the parametrization 



cos A 



sin A = 



A, 



1^1 



VA 2 P + \A S \ 



(8) 



where A is in the range [0, ir/2}. The term cos 2 A (sin 2 A) 
represents the fraction of the P-wave (S-wave) intensity 
at that value of vtik-k- The distribution (Eq. (6)), nor- 
malized so that at any fixed tuk-k the integral over the 
angular variables yields unity, is given by 

gs+p(w, mKir, -A, A) = cos 2 Xg(u:;A) + sin 2 A fr(u) 



-sin2A 
2 



/ 8 (u>)cos(<5||-<y s )|A||| 
+f 9 {u)MS±-Ss)\A±\ 

+/io(w) cos(6 s - S )\A \ 



(9) 



In Eq. (9), the dependence of gs+p on ttik-k follows from 
that of A and of the strong phases (i =_L, || , 0, S). We 
see that at a given value of vtik-k the equations are in- 
variant under the transformation 



(5\\ - 5 ,5 ± - S ,5 S - 6 ) <— 
(S - S\\,n + S — Sj_,S 



We will use the change of the S-P relative phase in the 
region of the K * (892) to resolve this ambiguity. 

The phase of a weak decay amplitude is determined 
by phases introduced through the weak interaction itself, 
that is from the CKM matrix, and by strong final-state 
interactions. If in the decay B — > J/ipKir the J/4> were 
known not to interact with the Kix system, Watson's 
final-state interaction theorem [28] would guarantee that 
the phases for the P-wave and S'-wave final states would 
be simply the corresponding phase shifts in P-wave and 
S'-wave Kit scattering at the appropriate invariant mass, 
taking Kn scattering to be elastic in this range. How- 
ever, we know this is not exactly the case, for if it were, 
the three individual P-wave amplitudes would be rela- 
tively real (relative phases or it). This is not the ex- 
perimental result, as we shall show. Nonetheless, we will 
provisionally adopt the assumption that the interactions 
with the J/ip are small, and in particular that they do 
not change significantly with mjf ff . We then anticipate 
that the difference 6s — 60 will behave much like the dif- 
ference 6(Kir(L = 0)) — 6(Kn(L = 1)), where we restrict 
ourselves to the 7=1/2 channel, which is produced in 
the B — ► J/ip Ktt decay. According to Wigner's causality 
principle [29] , the phase of a resonant amplitude increases 
with increasing invariant mass. Since the Kir, 1=1/2 
P-wave phase shift increases rapidly in the vicinity of the 
if* (892), while the corresponding S-wave increases only 
very gradually, we expect 6s — 6q, 6s — 5±, and 6s — 611 
to fall rapidly with increasing niK* in this region. 



C. Time-Dependent Angular Distribution 

The time-dependent angular distribution for a B° me- 
son produced at time t = decaying as B° — ► J/ip K*° 
(K*° — ► if' 7T°) at proper time t has the same form as 
in Eq. (1) but with time-dependent amplitudes A(t): 



. 9 (o;;A(i),sin2/3,cos2/3) = I— d J 

T dt d cos 6k* d cos tr (- 



<Ptr 
(11) 



6 S ). (10) 



Under the hypothesis of no direct CP violation in the de- 
cay, i.e. |^4j(0)| = |Aj(0)|, i = 0, ||,_L, the corresponding 
terms that enter Eq. (11) are [16, 17, 27] 
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(-) 

A) (t) 



(-) 

A\\ (t) 



(-) 

A ± (t) 



/(-) (-) ' 
3m U,| (i) i4±(t) 

/(-) (-)* N 
ReU|| (t) A (t) 

/(-) (-)* N 
3m A|_(t) A (t) 



= e- r °*|A | 2 



= e- r °*U„| 2 



e - r "*|A±| 2 



1 + sin 2/3 sin Ami 



(-) 

1 + sin 2/3 sin Am t 



(+) 

1 — sin 2/3 sin Amt 



(-) 



e r °* | A\\ \\A±\ [sin(5j_ - S\\ ) cos Am t - cos(5± - S\\ ) cos 2/3 sin Am t ] , 



e-^l^lPol cos(5||-5 ) 

(-) 



1 + sin 2/3 sin Amt 



= + e ot \A±\\A \ [sin(5j_ - S Q ) cos Amt - cos(<5j_ - So) cos2/3sin Amt ] 



(12) 



for an initial P° (B ) meson. The mass difference between the two neutral B mass eigenstates is Am, and To is 
the common neutral P-meson decay rate, neglecting the lifetime difference between these mass eigenstates. The 
expression for the differential decay rate can be recast in the following form [27] : 



g n (<*>, t; A, sin 2/3, cos 2/3) = -^-e~ rot A(uj; A) x 



1 + rj 



V(oj;A) . A (S(u;A) . nn C(u;A) 
cos Amt + sinAmt ( /, A { sin 2/3+ / / cos 2(3 



A(u;A) 



A(u>;A) 



with r] = +1 (rj = —1) for an initial B° (B°) meson. The angular terms A, V, S, and C arc 

A(e>; A) = h(u>)\A \ 2 + / 2 («)|A|| | 2 + / 3 (u>)|Ax| 2 + / 6 («)|A)||A|| | cos(J|| - <5 ), 

= / 4 H|A||P ± |sin(^-<5||) + / 6 (u;)|Ao||A ± |sin((5x-^), 

5(w;A) = / 1 (a;)|ylo| 2 + /2(^)|^||| 2 -/3(^)|^| 2 + /5M|Ao||A|||cos( ( 5|| - 6 ), 

C(u>;A) = -/ 4 (w)|^||||i4x|cos(5x-5||)-/ 6 (u;)|i4o||i4±|cos(<5x-5o). 

The time-dependent asymmetry in the decay then reads 

g + i (w,t; A, sin 2/3, cos 2/3) - t; A, sin2/3, cos 2/3) 



a(w, f ; A, sin 2/3, cos 2/3) 



3+i(a>, i; A, sin 2/3, cos 2/3) + g-i(v, t; A, sin 2/3, cos 2/3) 

P(u>;A) . . (SUa;A) . nn C(u>;A) 
cos Amt ,) .{ + sin Amt ( t ) ,{ sin 2/3+ v / cos 2/3 



^(u>;A) 



(13) 



(14) 



(15) 



This reduces to the usual expression for decays to CP 
eigenstates when only the CP-even (A , An) amplitudes 
are nonzero or when only the CP-odd (A± ) amplitude is 
nonzero. We now examine the terms on the right hand 
side of Eq. (15): 

• The cos Ami term makes the smallest contribu- 
tion to g v (<*>, t; A, sin 2/3, cos 2/3) because the dis- 
tribution of values taken by V(u); A)/ A(u); A), as 
shown in Fig. 2(a), peaks at zero. 

• The sin Am t term has explicit dependence on both 



sin 2(3 and cos 2/3: 

o The usual sin(Ami) sin 2/3 factor is weighted 
by the angular term S(u>; A)/A(u; A), which 
can take values between —1 and +1, and 
whose distribution is shown in Fig. 2(b). This 
distribution reduces to one peak at +1 or — 1 
for a pure CP-even (\A±\ = 0) or CP-odd 
(| Ax | = 1) decay, respectively. 

o The cos 2(3 contribution is characteristic of a 
vector-vector channel. This contribution ap- 
pears only via the interference terms involving 
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FIG. 2: (a) The distribution of V(u; A)/A{w; A), (b) 
S(u:; A)/A(u; A) and (c) C(w; A)/A(u>; A), where .4, 7>, 5 
and C are defined by Eq. (14), for a set of events generated 
according to the amplitudes A corresponding to the BABAR 
values in Table I. 

the CP-odd amplitude A± and the CP-even 
amplitudes A and A\\ (Eq. (14)). The an- 
gular term C(uj; A)/A(w, A) takes values in a 
range smaller than [—1, +1] (Fig. 2(c)), whose 
bounds depend on the amplitudes and phases. 
The distribution of this angular term tends to 
peak at zero (Fig. 2(c)), inducing some loss in 
sensitivity to cos2/3 compared to that to sin2/3. 

o The sin 2/3 and cos 2/3 contributions are distin- 
guished by the angular information only. 

o From the orthogonality of the angular func- 
tions S and C (Eq. (14)) and the angular sym- 
metry of g v (Eq. (13)), the sin2/3 and cos2/3 
parameters, if regarded as independent quan- 
tities, are uncorrelated in a fit of the differ- 
ential decay rate (Eq. (13)), in the limit of 
infinite statistics and in the absence of exper- 
imental effects. 

Under the transformation (<5|| — 5o,<5_l — S ) — > (So — 
<5ii , 7r + <5o — 5±_), A, V, and S are unchanged, while C 
changes sign, showing that the ambiguity in the strong 
phases translates into an ambiguity in the sign of cos 2/3: 

(5\\ - 6 Q , 5± - S , cos 2/3) < — > 

(5 o -5pn + 5o-5±,-coB20). (16) 

II. THE BABAR DETECTOR 

A detailed description of the BABAR detector is pre- 
sented in Ref. [30]. Charged particles are detected with 
a five-layer, double-sided silicon vertex tracker (SVT) 
and a 40-layer drift chamber (DCH) with a hclium- 
isobutane gas mixture, placed in a 1.5-T solenoidal 
field produced by a superconducting magnet. The 
charged-particle momentum resolution is approximately 
(Spt/pt) 2 = (0.0013p T ) 2 + (0.0045) 2 , where p T is the 
transverse momentum in GeV/c. The SVT, with a typ- 
ical single-hit resolution of 10 /im, measures the impact 
parameters of charged-particle tracks in both the plane 
transverse to the beam direction and along the beam. 



Charged-particle types are identified from the ionization 
energy loss (dE/dx) measured in the DCH and SVT, and 
from the Chcrcnkov radiation detected in a ring-imaging 
Cherenkov device. Photons are detected by a CsI(Tl) 
electromagnetic calorimeter (EMC) with an energy res- 
olution a(E)/E = 0.023 • {E/GeVy 1 / 4 e 0.019. The 
return yoke of the superconducting coil is instrumented 
with resistive plate chambers (IFR) for the identification 
and muons and the detection of neutral hadrons. 



III. EVENT RECONSTRUCTION AND 
SELECTION 

The event selection is similar to that used in our pre- 
vious analysis [19]. Multihadron events are selected by 
demanding a minimum of three reconstructed charged 
tracks in the polar angle range 0.41 < 8i a b < 2.54 rad. A 
charged track must be reconstructed in the DCH, and, if 
it does not result from a K ° decay, it must originate at 
the nominal interaction point within 1.5 cm in the plane 
transverse to the beam and 10 cm along the beam. Events 
are required to have a primary vertex within 0.5 cm of 
the average position of the interaction point in the plane 
transverse to the beamline, and within 6 cm longitudi- 
nally. Electromagnetic depositions in the calorimeter in 
the polar angle range 0.410 < 9i a b < 2.409 rad that 
are not associated with charged tracks, that have an en- 
ergy greater than 30 MeV and that have a shower shape 
consistent with a photon interaction are taken as neu- 
tral clusters. We require the total energy for charged 
tracks and photon candidates in the fiducial region to be 
greater than 4.5 GeV. To reduce continuum background, 
we require the normalized second Fox- Wolfram moment 
i?2 [31] of the event, calculated with both charged tracks 
and neutral clusters, to be less than 0.5. 

Charged tracks are required to be in regions of polar 
angle for which the particle identification (PID) efficiency 
is well-measured. For electrons, muons, and kaons the 
acceptable ranges are 0.40 to 2.40, 0.30 to 2.70, and 0.45 
to 2.50 rad, respectively. 

Candidates for J/ip mesons are reconstructed in the 
e + e~ and /i + /i~ decay modes, from a pair of identified 
leptons that form a good vertex. A Loose [32] identifica- 
tion condition is required for each muon. (The number 
of interaction lengths it traverses in the EMC and IFR 
must be consistent with the expectation, as must be the 
average number of hits in each layer of the IFR; the IFR 
hits and the track extrapolation must match; the energy 
deposition in the EMC must be small.) A Tight con- 
dition [32] is required for each electron. (The measured 
dE/dx must be consistent with expectations; the energy 
deposition in the calorimeter must be consistent with the 
momentum measured in the drift chamber, and the lat- 
eral shower shape must be consistent with an electromag- 
netic shower.) Electrons that have no EMC information 
are selected on the basis of dE/dx information alone. For 
J/ip — > e + e" decays, where an electron may have radi- 
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ated one or several Bremsstrahlung photons, the missing 
energy is recovered by identifying EMC clusters with en- 
ergy greater than 30 MeV lying within 35 mrad in polar 
angle and 50 mrad in azimuth of the electron direction 
projected onto the EMC. The lepton-pair invariant mass 
must be between 3.06 and 3.14GeV/c 2 for muons, and 
between 2.95 and 3.14GeV/c 2 for electrons. This corre- 
sponds to a ±3<t interval for muons, and accounts for the 
partially recovered radiative tail due to Bremsstrahlung 
for electrons. 

A candidate K® consists of a vertexed pair of 
oppositely-charged tracks with invariant mass between 
489 and 507MeV/c 2 , when interpreted as pions. The K° 
flight length must be greater than 1 mm, and its direc- 
tion must form an ang le with the K° momentum vector 
in the plane perpendicular to the beam line that is less 
than 0.2 rad. 

Neutral clusters, as defined above, are used as photon 
candidates for the reconstruction of ir° —> 77 decays. A 
7T° candidate consists of a pair of photons with invariant 
mass in the interval 106 to 153 MeV/c 2 , and a total energy 
greater than 200 MeV. 

The J/ip , K®, and 7r° candidates are constrained to 
their corresponding nominal masses. Except in the anal- 
ysis that includes an S'-wave contribution, K* candidates 
must have a Kir invariant mass within 100MeV/c 2 of the 
nominal if* (892) mass. 

The J/ip and K* candidates are combined to form 
B — > J/ipK* candidates. It may happen that a gen- 
uine J/ipK* event is reconstructed incorrectly, most of- 
ten with the true J/ip , but with a wrongly reconstructed 
K* . This happens mainly for B candidates with a daugh- 
ter 7T°, with cross- feed (CF) from the companion channel 
with a 7r ± , or self cross-feed (SCF) when the genuine tt° 
is incorrectly reconstructed with at least one wrong pho- 
ton candidate. The (S)CF is reduced by demanding, for 
channels with a 7r° in the final state, that cos9k* < 0.7, 
where 6 k* is the -£T*-decay helicity angle (see Fig. 1). In 
addition, as was done in Ref. [27], if a single event can 
be reconstructed in two different K* modes and if one 
reconstruction uses a ir° and the other does not, the re- 
construction without a 7T° is retained. This reduces the 
cross-feed by 75% for a 1% relative loss in signal effi- 
ciency. In modes with a n , no cos 9k* cut is applied. 

Two kinematic variables are used to further discrim- 
inate against incorrect B candidates. The first is the 
difference AE = E* B — E^ cam between the candidate- B 
energy and the beam energy in the T(4S) rest frame. 
In the absence of experimental effects, reconstructed sig- 
nal candidates have AE = 0. The second is the beam- 
energy-substituted mass mEs = (-^oxp ^Pb) 1 ^ 2 where, in 
the laboratory frame E exp = (s/2 + PB-Pi) / Ei is the B- 
candidate expected energy, pg, its measured momentum, 
(Ei,pi), the e + e~ initial-state four-momentum, and t/s 
is the center-of-mass energy. For the signal region, AE is 
required to be between —70 MeV and +50 MeV for chan- 
nels involving a 7r°, and within 30 MeV of zero otherwise. 
If several B candidates are found in an event, the one 



TABLE II: Event yield and purity, estimated from a fit to 
the 771ES distribution (Fig. 3), with a Gaussian signal distri- 
bution and an ARGUS threshold function [33] describing the 
combinatorial background. The spectra are integrated over 
the range wies > 5.27 GeV/c . No correction for cross-feed is 
made since these numbers are not used in the actual analysis; 
rather they provide an indication of the purity of the data 
sample. 



Channel 


Yield 


Purity (%) 




131 ± 14 


81.6 




2376 ± 51 


95.8 




670 ± 27 


95.7 


J/ip (K^TT ) 


791 ± 33 


85.0 



with the smallest \AE\ is retained. 

The toes distributions for the B — > J/ip Kir candidates 
are shown in Fig. 3. Corresponding signal yields and pu- 
rities are given in Table II. These results are obtained 
from fits to the Toes distributions using a Gaussian dis- 
tribution for the signal and an ARGUS shape [33] for the 
combinatorial background of the form 

a(^Es) = a m-Es\/ 1 ~ (mEs/m ) 2 x 

e £(l- Ks/mo)^ ( 17 ) 

for toes < m o, where too represents the kinematic up- 
per limit and is fixed at the center-of-mass beam energy 
-^beam = 5.291 GeV. The parameter £ determines the 
shape of the spectrum. 

With the signal region defined by toes > 5.27GeV/c 2 
and the above AE ranges, the B reconstruction efficien- 
cies, summed over J/ip — > e~e~ and J/ip — > 
are (9.6 ± 0.1)%, (24.5 ± 0.1)%, (19.7 ± 0.2)%, and 
(12.5 ± 0.2)% for the modes K° s tt°, tt^, K° s tt ± , 
and K ± tt° , respectively. The composition of the re- 
maining background events is given in Table III. The 
contribution of B candidates with a fake J/ip candidate 
is less than 2%. 



IV. ANGULAR ANALYSIS 

The parameters 6a, 4>a (Eq. (5)), <5|| — S and S± — 
S of the angular-dependent time-integrated decay rates 
are determined using a simultaneous unbinned likelihood 
fit to the three flavor-eigenstate B — > J/ipK* channels: 
J/ipiK^), J/ipiK ^), and J/ip (l^V). The PDF, 
before accounting for the experimental effects described 
below, is given by Eq. (1). We first consider only the 
P-wave amplitudes; the effect of the S'-wave amplitude 
is discussed in Sec. IV D. The B flavor is taken into 
account in the fit through the relations (A = Aq, = 
i4||, A± = —A±) as explained in Sec. I. 
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TABLE III: The expected number of background events for each decay mode in the signal region, in an on-peak 81.9 fb _1 
sample. The contribution from continuum is estimated using a 9.6 fb _1 off-peak data sample. The BB contribution is estimated 
using a fully-simulated sample of generic BB decays equivalent to 72 fb _1 (with the inclusive J/ip events removed from the 
sample). The inclusive J/ip contribution is estimated using a fully-simulated sample equivalent to 91 fb _1 (B — > J/ip K* (892) 
events removed). 



Background 
Source 


J/tP{KU q ) 




B Decay Mode 


J/iPiK+iv ) 


Continuum 


0.2 ±0.1 


1.2 ± 0.4 


0.1 ±0.1 


0.7 ±0.4 


Generic BB 


0.2 ±0.1 


1.2 ± 0.2 


0.3 ±0.1 


1.2 ± 0.3 


Inclusive J/tp 


22 ±5 


126 ± 12 


38 ±7 


135 ± 12 
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FIG. 3: The jbes distributions for the AE intervals described 
in the text with overlaid Gaussian and ARGUS fit functions, 
for J/ip K-k candidates in data. 



A. Acceptance Correction 



The acceptance correction is applied as in our previous 
measurement [19]. We perform an unbinncd likelihood fit 



g obs (u J] A)=g(u;A) 



(18) 



where g(uj- A) is given by Eq. (1), e(u;) is the angle- 
dependent acceptance, and 

(e)(A) = j g(u>-A)e(u>)&u> (19) 

is the average acceptance over the event-weighted phase 
space, which depends on the amplitudes A, and which 
ensures the normalization of g obs . 

In the case of the J/ipK* channels studied here, the 
presence of cross-feed from the companion channels, 
which have, as a consequence of isospin symmetry, the 
same A dependence as that of the signal, must be 
taken into account. The observed PDF for channel b 
(b = K ± TT*,K s n ± ,K ± n°) is then 



g b <° bs (u;;A) = g( U ;A) 



e b {u) 



with 



a 



(20) 

(21) 
(22) 



and a = K° s n° , K^, K^tt* , K ± n°. In the above ex- 
pressions, the Ai.,.6 terms are (see Eq.(l)) 





= \Ao\ 2 , 


A 2 


= lV> 


A 3 


= l^l 2 , 


Ai 




A 5 


= Re(^ii), 


A 6 


= <3m(A*A ± ), 



(23) 



and F a is the fraction of mode a in B ^ J/tp K* decays 
(with Y, a F a = !)■ We assume that B{T{AS) -» B°B°) = 
B(Y(AS) -> B+B~), T(K*° -» K+tt-) = 2x T(K*° -» 
K°n°), and T(K*+ -> K%+) = 2 x T(K*+ -» K+tt ). 
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The measured values [34] of the branching fractions for 
the decays B° -» J/ipK*° and B+ -» J/ipK*+ are used. 
The angular functions /fe(u>) (fc = 1 . . . 6) have been 
defined in Eq. (2) and e a ^ b (uj) is the probability for 
an event generated in channel a and with angles u> to 
be detected as an event in channel b. Finally, e b (u>) 
is the efficiency for reconstructed channel b considering 
B — > J/ipK* channels as a whole, that is counting cross- 
feed events as signal. The $^ are the fk{<*>) moments of 
the "whole" efficiency e b . 

The angular resolution has been neglected, even for 
(self)cross-feed events. Also the possibility of doubly 
misidentifying the daughters of the K*° — > K + ir~ candi- 
date (K— 7r swap) is not taken into account. The induced 
biases have been estimated with Monte Carlo (MC) based 
studies, and found to be negligible. Under these two ap- 
proximations, the acceptance s b (u>) can be factorized as 
in Eq. (20), and only the coefficients <& b k are needed. 

The coefficients $^ are computed with exclusive signal 
MC samples obtained using a full simulation of the ex- 
periment [35-37] . Particle identification efficiencies mea- 
sured with data control samples are used to adjust the 
MC simulation to represent the actual behavior of the de- 
tector. Separate coefficients are used for different charges 
of the final state mesons, in particular to take into ac- 
count the charge dependence of the interaction of charged 
kaons with matter, and any other possible charge asym- 
metry of the detector. Writing the log-likelihood function 
for a pure signal sample we have, for each channel b, 

N evt 

L b (A) ee ^In^^A)) 

»=i 

= A)) -N evt \n PTA(A)<i>M 

i=i \ k J 

N evt 

+ ]rin(e h K))- (24) 
i=i 

where u>i represents the measured angular variables for 
event i, and N evt is the total number of signal candidates. 
When maximizing L b (A), the third term, which does not 
depend on the amplitudes, can be ignored [19]. 

B. Background Subtraction 

In our previous measurement [19] of the decay ampli- 
tudes, it was assumed that the combinatorial background 
could be taken into account with an expansion in the 
same basis functions as the signal. The systematic bias 
due to neglecting the missing components of the back- 
ground angular distribution was checked with MC-based 
studies. 

Here, we use an improved background correction 
method in which events from the toes sideband are added 
to the log-likelihood that is maximized, but with a neg- 
ative weight. 



The sample of N evt events selected in the signal region 
contains ns signal events and iib background events, so 
that N evt = ns + Ub- The values of ns and ns are un- 
known a priori. The quantity we would like to maximize 

ns 

is J2 ln {g h '° hs {ui; A)), while we have 

i=l 

N evt 

L b (A) ee J2^(g b - obs (u> i; A)) 

i=l 

ns 

i=l 

n B 

+ gln^^A)). (25) 
i=i 

Note that the same PDF appears for both the signal 
events and the background events: the PDF g b > obs of the 
signal. We use a pure sample of background events to ob- 
tain an estimate of the second term. This sample is from 
the mES sideband region 5.20 < toes < 5.27GeV/c 2 , 
which contains Nb events. It can be shown that maxi- 
mizing the modified expression L b ' , where 

N evt 

L b \A) ee J2Hg b - obs (^A)) 

i=i 

N B 

Nb k=i 

ns UB 

= M<7 M6S K; A)) + ]T ln(g b '° bs (u)j;A)) 

i=l j=l 

N B 

- ^Eln(fl t,o6 > fc ;A)), ( 26 ) 

k=l 

yields an unbiased estimator of the true parameters if 
is an unbiased estimator of ns- The quantity fis is 
obtained by fitting the data from the toes sideband and 
signal regions with a combination of an ARGUS and a 
Gaussian function. Since there is no peaking background 
contribution in the signal region, we take for h b the por- 
tion of the ARGUS fit that falls in the signal region. 

As L bl is not a log-likelihood, the uncertainties yielded 
by the minimization program Minuit [38] are biased es- 
timates of the actual uncertainties. An unbiased esti- 
mation of the uncertainties is described and validated in 
Appendix A. 

With this pseudo-log-likelihood technique, we avoid 
parametrizing the acceptance as well as the background 
angular distributions. This technique and the combined 
(toeS) angular) likelihood fit used in Ref. [19] rely on the 
assumption that the angular behavior of the combinato- 
rial background is the same in the toes signal region and 
sideband. The possible bias related to this assumption is 
discussed in the next section. 
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TABLE IV: Bias (in units of 1CT 3 ) observed in fits for the 
individual K* channels and the combined channel, based 
on parametrized Monte Carlo, taking as input the values of 
the amplitudes from Ref. [19]. The upper part of the table 
presents results for the fitted quantities 9a,4>a, 6\\ — So, and 
S± — So , all expressed in radians. The lower part presents re- 
sults for the amplitude moduli squared, which are computed 
from 6a and RA- 



TABLE V: Systematic uncertainties in the relative phases 
( rad) and in the amplitude moduli squared, for the three 
K* channels combined. 







Bias (10 










A S 7T 




all K* 


6a 


2.9 ± 1.0 


-1.3 ± 1.9 


-8.7 ± 1.9 


-0.2 ±0.8 




3.3 ±3.5 


13.0 ±6.6 


5.5 ±6.4 


5.5 ±2.8 


5\\ - 8 


-34.9 ± 7.8 


-19.2 ±14.7 - 


54.5 ± 13.9 


-36.2 ±6.2 


S± — So 


-29.3 ±6.4 


-7.8 ±11.8 - 


29.0 ± 11.4 


-25.2 ±5.0 




-2.9 ± 1.0 


1.3 ± 1.9 


8.5 ±1.9 


0.2 ±0.8 




0.4 ± 1.5 


-5.9 ±2.8 


-7.2 ±2.7 


-2.2 ± 1.2 




2.5 ± 1.4 


4.6 ±2.7 


-1.3 ±2.6 


2.1 ± 1.1 



C. Validation 

The complete fit scheme, including acceptance and 
background corrections as described above, has been val- 
idated with a BB Monte Carlo sample equivalent to an 
integrated luminosity of 590 fb -1 , produced with a full 
simulation of the BABAR detector (based on GEANT4 [35- 
37]). In this sample only events with a true J/tp — > (2 
decay with center-of-mass momentum V*ji^, greater than 
1.3 GeV/c are simulated. This momentum cut is not ap- 
plied in the analysis. It does not affect the signal region 
(toes > 5.27GcV/c 2 ), but means that only a subset of 
the events in the toes sideband region is included. 

An additional study has been performed with a larger 
sample generated with a parametrized simulation from 
the same event generator [35] with resolution effects and 
efficiencies incorporated. The equivalent integrated lu- 
minosity of this sample is 16 ab _1 . 

The results of the two simulations are found to be com- 
patible with each other. No statistically significant bias 
is observed with the full simulation. However, the high- 
statistics fast simulation shows small biases in the fitted 
parameters (Table IV). A contribution to the systematic 
uncertainty is derived from these biases in Sec. IV D. 



D. Systematic Uncertainties 

Table V summarizes the systematic uncertainties for 
the measurement of the amplitudes. The sources of un- 
certainty we have considered are described here. 

1. "cm. energy": The center-of-mass energy, which 
defines the toes endpoint spectrum, enters as the 
parameter to of the ARGUS function (Eq. (17)). 
The value (5.291 GeV/c 2 ) is changed by ± 2 
MeV/c 2 (uncertainty on the beam energy in the 





Source 


S\\ - So 


<5± — So 


\Ao\' 2 


\^\" 


\A±f 


1. 


cm. energy 


0.003 


0.005 


0.0018 


0.0002 


0.0016 


2. 


Bkg. shape 


0.002 


0.003 


0.0012 


0.0001 


0.0011 


3. 


BR 


0.000 


0.001 


0.0001 


0.0001 


0.0001 


4. 


MC stat. 


0.014 


0.008 


0.0027 


0.0023 


0.0024 


5. 


Fit bias 


0.036 


0.025 


0.0002 


0.0022 


0.0021 


6. 


PID 


0.005 


0.004 


0.0013 


0.0023 


0.0019 


7. 


S wave 


0.033 


0.038 


0.0029 


0.0025 


0.0005 




Total 


0.052 


0.046 


0.0048 


0.0046 


0.0042 



cm. frame) and the largest deviation from the 
nominal fit result is taken as the estimate of the 
systematic uncertainty. 

2. "Backgrounds shape" : The ARGUS function shape 
parameter £ (Eq. (17)), fitted to the toes spectrum, 
is changed by ±1 standard deviation and the largest 
deviation from the nominal fit result is taken as the 
estimate of the systematic uncertainty. 

3. "BR": The relative branching fractions of neu- 
tral and charged B mesons to J/ip K* affects the 
amount of cross-feed. The branching ratios are 
changed independently by ±1 standard deviation 
[34] and the largest difference is taken as the sys- 
tematic uncertainty. 

4. "MC stat." : The finite Monte Carlo sample size in- 
duces a limited knowledge of the coefficients § b k . 
This effect is evaluated by splitting the original 
Monte Carlo sample into ten equal-sized subsam- 
ples, each of which is used to compute the <fr b k co- 
efficients. These coefficients are then used for ten 
angular fits on the data, all differences being thus 
due to differences of the <& b k coefficients. For each 
fitted parameter, the standard deviation is com- 
puted, and divided by vTO to estimate the "MC 
stat." effect due to the original Monte Carlo finite 
size. 

5. "Fit bias": Biases are observed in validation stud- 
ies (Table IV). The observed bias is used as an 
estimate of the systematic uncertainty. 

6. "PID" : The efficiency of the particle identification 
has angular dependence. The induced effect on the 
fitted parameters is corrected by the acceptance- 
correction scheme. Imperfect knowledge of the 
particle-identification efficiency will result in a bias. 
A conservative estimate of the systematic uncer- 
tainty is obtained by using acceptance-correction 
factors for different beam conditions, correspond- 
ing to the years 2000, 2001, and 2002, and using 
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the largest differences as estimates of the system- 
atic uncertainties. 

7. "S 1 wave" : An additional fit is performed with the 
gs+p PDF (see next Section). The full gs+p-to- 
gp shift is used as a conservative estimate of the 
contribution to the systematic uncertainty, as was 
done in Ref. [19]. 



E. Results of the Angular Analysis 

Table VI summarizes the results of the fit to the angu- 
lar distribution. The fitted values for each channel and 
for each year of data collection (with statistical uncer- 
tainties only) are shown. Keeping in mind the two-fold 
phase ambiguity (Eq. (4)), we obtain 

5\\- 8 = (2.73 ± 0.10 ± 0.05) rad, 

5±-5 = (0.18 ±0.07 ±0.05) rad, 

\A a \ 2 = 0.566 ±0.012 ±0.005, 

|A||| 2 = 0.204 ±0.015 ± 0.005, 

\A ± \ 2 = 0.230 ±0.015 ±0.004, (27) 

where the correlation matrix of the fitted parameters 
{0a, <j>A, S\\ - S , S± - 6 ) (Eq. (5)) is 

/ 1.00 0.00 -0.04 +0.04 \ 

1.00 -0.23 -0.09 

1.00 +0.65 
V 1.00/ 

Angular distributions for the three channels are shown 
in Fig. 4. A forward-backward asymmetry is clearly vis- 
ible in the comparison of the distributions of cos 6 k* for 
(pure P-wave) MC, generated with the amplitudes found 
in the data, and for the data samples themselves. This 
is due to S-P interference. 

In a series of 168 simulated experiments of the same 
size as the data sample, we find that the probabil- 
ities for obtaining a larger likelihood than that ob- 
served for the data are 11%, 47%, 58%, and 25% for 
the iC ± 7r =F , K^n^, K ± tt°, and combined samples, respec- 
tively. 

The results for J/tp K*° and J/tp K*± decays are found 
to be compatible with each other (Table VI); this con- 
firms the expectation of isospin symmetry. 

From Eq. (27), we note that 5n — So differs from tt by 
3.6 standard deviations and that 5± — Sq differs from 
by 2.0 standard deviations. In order to determine the 
uncertainty in Su — S±, the combined data sample is refit 
using <5|| — S± and S — S± as phase parameters. The 
resulting amplitudes and the value of So — S± were as 
before, and this refit yields 

5\\-6± = (tt- (0.60 ±0.08 ±0.02)) rad, (28) 

where the systematic uncertainties have been estimated 
as in Sec. IV D. The Sn —S± statistical uncertainty agrees 
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FIG. 4: Angular distributions. Histogram: Inclusive J /if) MC 
sample (p*j/^ > 1.3 GeV/c). Points: Data. The spectra are 
acceptance-corrected, background-subtracted, and normal- 
ized to the estimated yields (Table II). The visible forward- 
backward discrepancy in the cos 9k* distribution is due to 
the Kit S-wave amplitude present in the data, and absent in 
the MC sample. The related systematic uncertainties in the 
measurements of the decay amplitudes are listed in line 7 of 
Table V. 



with that expected from Eq. (27) taking into account the 
65% correlation between the S\\ — So and — 5 param- 
eters. The departure from n is 7.6 standard deviations, 
and this demonstrates quite clearly the presence of final- 
state interactions between the J/tp and the K* . 



V. RESOLVING THE STRONG PHASE 
AMBIGUITY 

In our earlier publication [19] we presented evidence 
for the presence of a Ktt S'-wave amplitude in the 1.1 < 
rriKir < 1-3 GcV/c 2 range. We study this S wave in more 
detail here, in particular its interference with the P wave 
in the vicinity of the if* (892) resonance. We then use 
this interference to resolve the strong phase ambiguity 
for the B — > J/tpK *(892) decay amplitudes, using the 
observations of Sec. IB. 

In the following we will denote the two strong phase 
solutions obtained in the analysis of Sec. IV based on a 
purely P-wave angular distribution, by: 

Solution I : (S\\ - <5 , S± - S ) ^ (2.7, 0.2), (29) 
Solution II : (S\\ - S , S± - S ) ^ (-2.7, tt - 0.2). 

(30) 

The Ktt mass requirement mentioned in Sec. Ill (ttik-k 
within 100MeV/c 2 of the nominal if* (892) mass) is now 
relaxed, and the whole kinematical domain for the Kit 
system from B — > J/tp Ktt decay is used. The rriK-n spec- 
tra are shown in Fig. 5. 
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TABLE VI: Values of |Ao| 2 , \A\\ | 2 , |A^| 2 , <5|| — <5o, and <5x — So, for subsamples of the data divided according to decay channel. 
The first uncertainty is statistical; the second, when given, systematic. Note that the phases are subject to a two-fold ambiguity 
(Eq- (4)). 

Sample \A \ 2 \A^\ 2 \A ± \ 2 8\\ - 5 ( rad) S± - S ( rad) 

K* tt^ 0.560 ±0.015 ±0.005 0.208 ± 0.019 ± 0.004 0.232 ± 0.020 ± 0.005 2.673 ± 0.121 ± 0.052 0.159 ± 0.084 ± 0.048 

K° s n ± 0.560 ± 0.028 ± 0.006 0.232 ± 0.034 ± 0.010 0.208 ± 0.034 ± 0.007 2.747 ± 0.220 ± 0.052 0.124 ± 0.174 ± 0.050 

if* 7T° 0.592 ±0.028 ±0.013 0.165 ± 0.032 ± 0.011 0.243 ± 0.036 ± 0.009 2.904 ± 0.287 ± 0.090 0.329 ± 0.176 ± 0.066 

Total 0.566 ±0.012 ±0.005 0.204 ± 0.015 ± 0.005 0.230 ± 0.015 ± 0.004 2.729 ± 0.101 ± 0.052 0.184 ± 0.070 ± 0.046 
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FIG. 5: The background-subtracted Kn invariant mass dis- 
tributions for J/tp Kit candidates in data. 



A. Probing the S—P interference 

We use the K ± n T sample since it is the largest sample 
and has the lowest background level. We split this sam- 
ple into Kn mass intervals so that each interval has ap- 
proximatly the same number of candidates. Equation (9) 
shows that the presence of both Kn P-wave and Kn S- 
wave amplitudes (i.e. A ^ and A ^ n/2) implies the 
presence of S-P interference. Before fitting the data to 
the distribution of Eq. (9), we check for the presence of 
such interference effects by evaluating the moments of 
the angular functions /s,g,io- The orthogonality of these 
functions is expressed by 



/ 



f i (u)f j (u)du = 5 ij K i (i - 8,9,10; 

j = 1,...,10), (31) 



with k 8 — Kg = 3/407T and k w — 3/4-7T. The moments 
are defined by 



(/») 



J gs+p(> 



u;m K7! ,A,\)fi(<jj)d<jj, (32) 



and are functions of nix-n- Using Eq. (32), we obtain for 
i = 8,9, 10: 

2 

— (is) = sin2Acos((5|| - 5 S )|A|||, 

Kg 

2 

— </ 9 ) = sin2Asin((5 ± - S S )\A±\, 
Kg 

— </io) - sin2\cos(5 s - 8o)\A \. (33) 

KlO 

The behaviour with tjikt: of the right side of Eq. (33) 
terms in data can be displayed by evaluating, in each Kn 
mass interval, the related moments. Their background- 
subtracted, acceptance-corrected distributions are shown 
in Fig. 6 for data. They show rapid variation near the 
position of the K * (892), where the phase of the P-wave 
changes most rapidly. Similar distributions obtained 
from inclusive J/tp MC samples, in which no interfer- 
ence between S and P waves is simulated, show values 
of the moments compatible with zero in the correspond- 
ing mass range. In addition, the fact that the moments 
(/s), (/9) ; (/io) show significant deviation from zero in 
the Ktt mass region above 0.8 GeV/c 2 is a clear indica- 
tion of the presence of an S'-wave Kn amplitude in this 
region, interfering with the P-wave amplitudes. 

We also note that the cos 9k* forward-backward asym- 
metry 



i-FB 



N{cos9 K * > 0) - 7V(cos6> K » < o) 



N(cos9 K * > 0) 
V3 
2 



N(cos9 K * < 0) 
sin2Acos((5s - $o)\A \ (34) 



is proportional to (/ 10 ) (Eq. (33)). The distribution of 
(Ao) (Fig- 6(c)) has a mean value of —0.14 ± 0.03 in 
the 0.8 — 1.0 GeV/c 2 Kn mass range, thus indicating a 
global cos 9k* backward trend in the i^*(892) region, as 
observed in Fig. 4. 



B. Fitting for S s — 5 

The S ± P angular distribution (Eq. (9)) is fit to the 
data in each Kn mass interval of Fig. 6 in order to ob- 
tain the values of A and 7 = S s — S . Separate fits 
are performed for the two possible strong phase solu- 
tions (Eqs. (29,30)). We fix the P-wave amplitudes to 
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FIG. 6: Measured values of (a) (f$) ■ 2/ks, (b) (/g) • 2//tg and 
( c ) (/io) ' 2/kio, defined in Eq. (33), as a function of mjfi, 
for the J/ip K ± -k^ candidates in data. The three distributions 
show a clear variation near the K* (892) region. 



the values obtained previously (Eq. (27)). The methods 
for acceptance correction and background subtraction de- 
scribed in Sec. IV are also applied here. Any variation 
of the acceptance with rriK-n is neglected. 

The fit results for the K^ir^ channel are shown in 
Fig. 7. Figure 7(a) shows the P-wave intensity namely 
cos 2 A • rig, and Fig. 7(b) shows the S*-wave intensity. 
The quantity ns is the estimated number of signal events 
in the given mass interval and is obtained from 

a fit similar to that in Fig. 3, but with the Gaussian 
parameters fixed to the values obtained there. 

Figure 7(c) shows the phase 7 as a function of vtik-k for 
the two solutions for the strong phases (Eqs. (29,30)). We 
see that the two solutions show opposite behavior in each 
mass interval, as they must (see Eq. (10)). The large 
excursion in the relative phase in passing through the 
if* (892) region supports our assumption that the phases 
of the decay amplitudes reflect the phases of the simple 
Ktt system. 

The full points of Fig. 7(c) are obtained with strong 
phases of "Solution II" , for which 7 is decreasing in the 
if* (892) region, as required for the physical solution. A 
conservative estimate of the discrimination between the 
two solutions is made by fitting for the slope d"f /dmK-n 
in the range 0.8 < mj(, < 1.0 GeV/c 2 ; we find 



FIG. 7: (a) The P-wave intensity times number of events; i.e., 
ns cos 2 A; (b) the S-wave intensity times number of events; 
i.e., ns sin A. These are the numbers of events that would 
be observed in each interval for the amplitude under consid- 
eration, if it were the only amplitude. The fraction of S-wave 
intensity integrated over the range 0.8 < mft < 1.0GeV/c 2 
is found to be (7.3 ± 1.8)%. (c) The evolution of 'y/n with 
rriKir, for the two sets of strong phases. The mirror symmetry 
described by Eq. (10) is clearly visible as 7 <-» 2n — 7. The 
error bars represent the statistical uncertainty in the fit of 
(A, 7) in the uik n interval considered. All distributions are 
from fits to J/ipK ± -K^' candidates in data. 



Solution I: 
Solution II: 



16.2 ± 2.7 rad/ GeV/c 2 
-16.2 ±2.7 rad/ GeV/c 2 



As they must, these two slopes have opposite values. The 
two fits have a x 2 P er degree of freedom of 1.6. Finally, 
interpreting Solution II as the physical solution, we ob- 
tain the unique result 



S [{ - 5 = (-2.73 ±0.10 ±0.05) rad, 

5±- 6 = (+2.96 ±0.07 ±0.05) rad, (35) 

i.e., the two relative phase values are approximately 
equal in magnitude but with opposite sign. 

It should be noted that this phase solution is not that 
selected in previous papers, nor in Eq. (27). 
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C. Examining the Moments 

The values of the moments (/§), (/ 9 ), and (fw) 
(Eq. (33)) are unchanged under the strong phases trans- 
formation Eq. (10), and, as such, do not allow us to dis- 
tinguish between Solution I and II; but we show here that 
their variation, in particular that of (fg), with m^, to- 
gether with the physical requirement that dj/drriK-K < 0, 
allows us to resolve the ambiguity, without relying on the 
explicit solutions displayed in Fig. 7(c). 

Since A is small and positive, sin2A > and from 
Eq. (33) we can write 



,1000 



d(/s) 
ArriK-K 
d(/ 9 ) 
dm K7T 
d(/io) 
dm^Tr 



+ sin((5|| - 5 Q - 7) 



d7 



- cos((5j_ - So - 7) 



drriKiT ' 
dj 



dmKir ' 



— sin 7 



d7 
dm 



Ktt 



(36) 
(37) 
(38) 



Given that the values for 5\\ and S± are close to or it 
(Eqs. (29,30)), we can approximate Eqs. (36,37) by 



d(/s) 
dm K7r 
d(/ 9 ) 



d7 

- cos((5|i - S ) sin j- , (39) 

dm K7T 

d'j 

- cos((5j_ — S ) cos 7- . (40) 



dm K7T dm Kn 
On Fig. 6(c) we observe, in the if* (892) region, that 



(ho) 
d(/io) 
dm Kn 



■ cos 7 

■ sin 7 



<0, 
d7 
dm K7T 



>0, 



meaning that 

d(/s) 
dm K7T 

d</ 9 > 



dm 



Ktt 



has the sign of cos((5|| — So), 
has the sign of cos((5j_ — S n ) 



d7 
drriK-K 



(41) 
(42) 

(43) 
■ (44) 



The variation of (fg) observed on Fig. 6(a) is compatible 
with Eq. (43), whichever strong phase solution is con- 
sidered (Eqs. (29,30)) and thus cannot distinguish be- 
tween the physical solution and the non-physical one. 
Figure 6(b) shows that d(/ 9 )/dmK 7r > 0, meaning that 
either 



cos(<5i_ — So) > and 
cos(<5j_ — So) < and 



d7 
dm K7I 

d7 
dm KlT 



>0, 



< 0. 



(45) 
(46) 



We note that cos(<5_l - So) > for Solution I (Eq. (29)), 
and cos((5 ± -S ) < for Solution II (Eq. (30)). The vari- 
ation of (fg) with uik-k provides thus the association of 
Solution II with the physical requirement d^jdraKix < 0, 
and of Solution I with the non-physical case d"f/dmK-K > 
0. This leads to select Solution II as the physical solution, 
consistently with the previous section. 
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FIG. 8: (a) Comparison of the K tj^ P-wave intensity with 
a Breit-Wigner lineshape, including a centrifugal barrier fac- 
tor, with world average parameters for if* (892) [34]. The 
lineshape is integrated in each mass interval (star markers) 
and compared with the measured intensity in that interval, 
after a minimun \ 2 fit of the overall normalization to the 
data in the 0.8-1.3 GeV/c 2 mass range. The \ 2 P er degree 
of freedom is 0.86. (b) Pull (i.e. difference of measured and 
expected intensities, normalized to the uncertainty) in each 
mass interval. 



D. Checking the (Kn) P - 



lineshape 



Figure 8 compares the P-wave intensity (as already 
shown in Fig. 7(a)) with a Breit-Wigner lineshape, in- 
cluding a centrifugal barrier factor, using the world av- 
erage [34] parameter values for if* (892). (The mass res- 
olution is about 3 MeV/c 2 and is negligible in its effect.) 
The overall normalization is fit in the 0.8-1.3 GeV/c 2 
mass range. The \ 2 P er degree of freedom is 0.86. The 
good agreement observed between the data and the Breit- 
Wigner lineshape suggests that the final-state interac- 
tions observed at the end of Sec. IV E, though statisti- 
cally significant, are not so great as to distort the line- 
shape. This is consistent with our hypothesis of small 
interaction between the J/ip meson and the Ktt system, 
made at the end of Sec. IB. 



E. Comparison with K p — > K 7r + (n) Scattering 
Results 



In Fig. 9 we compare the evolution of 7 observed in 
Fig. 7(c) with that obtained from the LASS measurement 
of K-p -> K-u+{n) scattering. The LASS points [39] 
(based on data from Refs. [22, 40]), represented as dia- 
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TABLE VII: Composition of the reconstructed B° — * 
J/ip K*°(Ks7T ) candidate sample in the region tues > 5.27 
GeV/c 2 . The backgro und fractions are estimated from an 
inclusive J/ip Monte Carlo sample with an equivalent inte- 
grated luminosity of 590 fb _1 with the requirement p*j^ > 
1.3 GeV/c. The first uncertainty is statistical; the second is 
systematic and is based on the uncertainty in the correspond- 
ing branching fraction [34] if it is known, and otherwise is 
based on a 50% uncertainty on the branching fraction used in 
the Monte Carlo generator. 



Fraction(%) Effective CP 



Signal 
Cross-feed 
B° - XciK° a 
Higher-mass K* resonances 



83.0 ±1.2 ±5.7 
3.2 ±0.5 ±0.3 
4.0 ± 0.6 ± 1.1 
3.5 ± 0.6 ± 1.8 



Non-resonant B° - 
Other B — > J/tpX 



J/ipKW 2.8 ±0.5 ±1.4 
3.5 ± 0.6 ± 1.8 





-1 
Oil 
Oil 
Oil 



FIG. 9: Comparison of the variation of 7 = 5s — So with 
tuktv for the J/ip K ir^ events, for "Solution I" (open points, 
Eq. (29)) and "Solution II" (full points, Eq. (30)), with that 
measured by the LASS experiment [22, 39, 40] (diamond 
markers) . 



monds, show the phase difference 

£s(/=i/2) - <5p(/=i/2) 

as a function of TOktt- Only the 7=1/2 amplitude is 
retained since this is the only one produced by the B — > 
J/ip Kir process. The LASS analysis takes into account 
the D wave, while the present analysis does not, but the 
D wave (#2(1430), T ~ 100 MeV) has an effect only at 
high rriK-K- An overall shift of 7r radian is added to the 
LASS phase difference measurements in order to match 
the sign of the forward-backward asymmetry observed 
with the B — ► J/ip Kir events. This shift does not modify 
the slope and general shape that are of interest here. 
The shift corresponds merely to changing the relative sign 
between the S and P wave amplitudes. The need for 
such a global shift is not surprising since the production 
processes are unrelated. We can see that the agreement 
between "Solution II" and LASS is striking (Fig. 9). 



particular, the strong phases <5|| — So and S± — So are fixed 
to "Solution II" (Eq. (30,35)), described in Sec. V. 

We examine each event with a B° — > J/ip K*°(K®ir°) 
candidate, indicated by Bqp, for evidence that the other 
neutral B meson, B tag , decayed as a B° or a B° (fla- 
vor tag, as described below). We also determine the de- 
cay proper-time difference At = tcp — t tagi which cor- 
responds to the variable t in Eqs. (11)-(15). To a good 
approximation c • At = Azj (73 ■ (3 ■ 7) where Az is the 
separation between the Bqp and B tag vertices along the 
e+e~ collision axis and 73 ■ -7 = (p e - — Pe+)/(2 • ttib)- 

The background level in the B Q — > J/ipK®n° sam- 
ple is higher than in the other B — > J/ipKit channels. 
In particular, some CP-violating backgrounds tend to 
peak in the signal region (rriES > 5-27 GeV/c 2 ), mak- 
ing the toes < 5-27 GeV/c 2 region a poorer represen- 
tation of the background behaviour than in the other 
B — > J/ip Kt: channels. In contrast with the method used 
for the angular analysis described above, the CP analy- 
sis is performed by the maximizing a likelihood function 
that contains the PDFs of both the signal and the back- 
ground. Only events from the signal region are used. 
Monte Carlo samples are used to determine the angular 
acceptance, and the background composition and angu- 
lar dependence, as described in the following Sections. 



VI. MEASUREMENT OF cos 2/3 

To measure cos2/3, we perform an unbinned maxi- 
mum likelihood fit to the differential decay rate as a 
function of proper time and the three angular variables 
(Eqs. (11,12)) for the B° -> J/^K*°{K°tt°) sample. 
The amplitude parameters |Ao|, \A±\, 5|| — So, and 
S± — So in Eq. (12) are fixed to those obtained by the an- 
gular analysis of the three high-statistics flavor-specific 
B — > J/tpK* channels, presented in Sees. IV and V. In 



A. Background Contributions 

The inclusive J/ip Monte Carlo sample is used to de- 
termine the composition of the B° -> J/ip K*°(K°tt°) 
candidate sample. The results arc shown in Table VII. 
Aside from the signal and cross-feed contributions, the 
dominant contributions are from B° — > \ c iK® and B° — > 
J/ipK** decays, where K** denotes higher-mass K* res- 
onances. The angle- and Ai-dependent PDF for each of 
these backgrounds is described in the next Section. 
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B. Acceptance Description 

The acceptance and the combinatorial background 
PDF are described by expansions in terms of products 
of spherical harmonics. The orthonormal basis functions 
used are 



y LRM {oj) = V2ttY L m(0^,x)Yrm{0k' 1 0), (47) 

where 0^,x are the helicity angles corresponding to the 
transversity angles 8 tr , 4>tr [27] and the Y\ m are spherical 
harmonic functions. These yLRM functions describe the 
partial waves involved in a "Scalar — > Vector X" decay, 
where X can be of arbitrary spin [14]. A function of u, 
say e(w), is expanded as: 



e ( W ) = ^LRM~y LRM (w) , 
LRM 

where the sums over L and R run, in principle, from 
to infinity, and the sum over M from — min(i, R) to 
+ mm(L, R). 

The moments of the acceptance are estimated from 
Monte Carlo simulation, with N gen events generated with 
PDF g, N b s events being finally selected: 



1 LRM — 



t{u)y* LRM (u})du3 



(49) 



The moments for the background PDFs are computed 
using the reconstructed events of the MC background 
sample distributed as b(oj): 



'LRM 



u)y*L RM (u)du; 



(50) 



i=l 



We note that the analytical expressions for the back- 
ground PDF and for the efficiency are not needed to com- 
pute these moments. The expansion is done up to rank 
L = R = 4 for the acceptance (signal distribution is of 
rank 2) and up to rank L = R = 6 for the background. 
These ranks are chosen to be large enough so that no 
significant deviation of the fitted parameters sin 2/3 and 
cos 2(3 is observed in high-statistics Monte Carlo samples 
when compared to the generated values. 



C. The B flav Sample 

The fit is additionally performed on a large sample 
of fully reconstructed B° decays to flavor eigenstates 
(Bfi av ) with decays B° — > D^^hr^ , where h — -K,p,a\ 
and B° -» J/ipK*° (K*° -> K^). These events are 



used to measure the parameters of the flavor-tagging al- 
gorithm and of the Ai-resolution functions. Flavor tag- 
ging performance is shown to be independent of the fully- 
reconstructed B meson and the At resolution is domi- 
nated by the vertex resolution of the incompletely recon- 
structed tagging B meson. Thus both tagging and At 
resolution can be studied with these large, well under- 
stood samples and the results applied to the channels of 
interest. 

The fully reconstructed B meson, i.e., Bfi av or Bcp, 
is denoted by B rec . 



D. Flavor Tag Determination 



We use a multivariate technique [3] to determine the 
(48) flavor of the B tag meson. Separate neural networks are 
trained to identify primary leptons, kaons, soft pions 
from D* decays, and high-momentum charged particles 
from B decays. Events are assigned to one of five mutu- 
ally exclusive tagging categories based on the estimated 
mistag probability and the source of the tagging infor- 
mation: Lepton, Kaon I, Kaon II, Inclusive and 
Untagged. The Untagged events are not used in this 
analysis. 

We determine the average dilution (D) and dilution 
difference AD, defined as 



D 
D 

(D) 
AD 



1 - 2w, 
1 - 2w, 
D + D 
~ 2~' 
D-D, 



(51) 



for each tagging category, where w (w) is the probability 
that a flavor tag determination is incorrect when the true 
tag is a B° (B°). The quality of the tag ging is expressed 
in terms of the effective efficiency Q = ^2 k £k(l — 2wk) 2 , 
where tk and Wk are the efficiencies and mistag probabil- 
ities, respectively, for events tagged in category k. The 
tagging performance is measured in a large data sample 
of fully reconstructed B decays. The effective tagging ef- 
ficiency is (28.1 ±0.7)% [3]. The tagging efficiency asym- 
metry between B° and B° has been studied [27, 41] using 
the full simulation of the experiment and has been found 
to be negligible for this analysis. 



E. Determination of At and At Resolution 

The proper-time difference At between the decays of 
the two B mesons in the event (B rec , B tag ) is determined 
from the measured separation along the collision axis, 
Az, between the B rec and the B tag vertices (Eq. (22) of 
Ref. [32].) The B tag decay vertex is obtained by fitting 
tracks that do not belong to the B rec candidate, imposing 
constraints from the B rec momentum and the beam spot 
location. The average At resolution is approximately 1.1 
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ps. We require that the measured proper-time difference 
between the B rec and the B tag decays satisfies \At\ < 
20 ps and that the estimated uncertainty in At, a At, 
which is derived from the vertex fit for the event, be less 
than 2.5 ps. 

The Ai-resolution function 1Z is represented by a sum 
of three Gaussian distributions (called the core, tail, and 
outlier components): 

lZ(6(At) = f core G(6(At); f-^core, O'core) \ 
ftailG(S(At); frail, Vtail) + 
foutlierG(8(At); ^outlier, Poutlier), (52) 



where G is the Gaussian function, 5(At) = At — At tIUC , 
Attruo is the actual decay time difference, and / core , ftaii, 
and f outlier the fractions of each component. 

For the width of the core and tail Gaussians (<7 cor e, 
&taii), we use the measurement uncertainty a At and allow 
separate scale factors S core and S ta a to accommodate an 
overall underestimate (Sk > 1) or overestimate (Sk < 1) 
of the errors for all events, so that cr core = S core <JAt and 

Vtail = Stail^At- 

The core and tail Gaussian distributions are allowed to 
have a nonzero mean (fj, C ore, frail) to account for charm 
decay products possibly included in the Stag vertex. In 
the resolution function, these mean offsets are scaled by 
the event-by-event measurement error a At to account for 



an observed correlation [32] between the mean of the 
8(At) distribution and the measurement error a At m 
Monte Carlo simulation. For the core we allow differ- 
ent means for each flavor-tagging category. One com- 
mon mean is used for the tail components. The third 
Gaussian has a fixed width <j ou tiier = 8ps and no offset 
(^outlier = 0); it accounts for fewer than 1% of events, 
typically due to incorrectly reconstructed vertices. 



F. Likelihood Function 

We maximize the log-likelihood given by 

Ltotal = L C P + Lfi av , (53) 



tag,c i—1 



/o&o + f±b± 
+ (i-(fo + f±))g K ^ a ' ohs 



(54) 



where /o and f± are the fractions of "neutral" and 
"charged" background in the CP sample, and bo and b± 
are the corresponding background PDFs, described be- 
low. The signal PDF g K °s*°< obs is 



gKs* > obs ( w , At; A, sin 2/3, cos 2/3) = c(uj)-^-A(uj; A) 

e n (At)-((D) 




, A .V(w;A) , A ,/«S(u>;A) . „„ C(u;A) 
c n {At) ) ' ( + s n (At) [ A- - A [ sin 2/3 + A ) - ~ ~ cos 2/3 



A(u,;A) 



i + C- 



AD 



, „ „ r 1 T ^d , „ „ 



(55) 



fe=l,2,3,5 



d fc=4,6 

r 



where £ labels the flavor of the tagging B meson (£ = 1) 
or B° meson (( = —1), and Xd = Am/T . Ak is defined 

b—K°7T° 

by Eq. (23), and <f>fe is the diagonal part of $ fe s de- 
fined in Eq. (22). Only e b ^ b ,b = K°w° (Eq. (22)) is 
considered because the cross-feed is treated separately 
here, in the background contribution, as it does not con- 
tribute to CP violation. The Ai-resolution function 1Z 
(Eq. (52)) appears in the following convolutions: 



e n (At) 
s n (At) 
cn(At) 



= e -r |At tr „ e | 
= -r \At trU e\ 



<lZ(6(At)), 



e ' sin(AmAt true ) <g> 11(6 (At)), 

e -r a \At truc \ C0S (A m At true ) <g> lZ(6(At)). 

(56) 



Most of the background is due to inclusive decays of 
B mesons to J/ip (see Table III). We account for back- 
grounds with the following PDF's: 

• Backgrounds from neutral- B decays (see Table VII) 
are parametrized with a form analogous to the one 
that describes J/tpK®, but with an effective CP 
eigenvalue, r\cp, and angle dependence b(u)): 



b x ( At, u;;sin2 ft, r) C p) 



E£ e -r„|A t | x 



1 + C-T - ) - C(-D)^CPsin2/3sin(AmAi) 



&(«) 



(57) 



For B° — > XciKg the angular dependence b(u)) is 
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estimated with the Monte Carlo, and parametrized 
using an expansion in y^ RM - For higher-mass 
K* resonances, non-resonant B° — > J/ipK^w , and 
other _B — > J/ipX sources, a flat angular depen- 
dence is used. 

• Backgrounds from charged _B decays (see Ta- 
ble VII) arc dominantly due to cross-feed from 
B ± — > J/i/jK* ± . They have a At distribution char- 
acterized by the decay rate T + . They are repre- 
sented by 



6±, c (At, W ) = -± e - r n 



|At| 



1 + 



6±(w) 



(58) 



The Bfi av sample, which is used to determine the tag- 
ging features, enters the log-likelihood through the Lfi av 
term (Eq.(53)), which is based on PDFs for "mixed" and 
"unmixed" events as is appropriate for these neutral B 
decays. The background PDFs include a zero-lifetime 
contribution, a contribution with an effective lifetime, 
and a contribution with an effective lifetime and an os- 
cillating factor. The signal PDFs are 



K x (At) cx 



1 + C~^j + u(D) cos(AmAi) 



(59) 



where u = 1 and u = — 1 for unmixed and mixed events, 
respectively. A complete description of the log-likelihood 
Lfiav term is provided in Ref. [32] (ln£ m ; x term in 
Eq.(6) of Ref. [32]). 

Finally the free parameters in the fit are (see Table XI) 

• sin 2(3 and cos 2/3 (2), 

• the parameters for the signal Ai-resolution function 

(8), 

• the tagging parameters for signal (8), 

• the parameters for the background Bji av At reso- 
lution function (3), 

• the parameters describing the composition of the 
background PDF for the Bfi av sample (13). 

In total there are 34 parameters. We fix Tq and Am to 
their world average values [34] . 



G. Validation 

The fitting scheme has been validated using the full 
simulation and the large parametrized MC samples men- 
tioned above. No statistically significant bias is observed 
(Table VIII). 

As a further cross-check, the data samples for the 
B^ — > J/ipK*" 11 , channels, which are not expected to 



TABLE VIII: Validation on full MC simulation [inclusive J/ip 
MC sample (p*j/^, > 1.3 GeV/c) and Bfi av samples] and large 
parametrized MC samples. The generated values of sin 2/3 
and cos 2(3 are 0.700 and 0.714, respectively. 



CP Sample 



sin 2(3 



cos 2(3 



FullMC (0.6 ah" 1 ) 0.61 ±0.16 0.20 ± 0.32 

Parametrized MC (16 ab^ 1 ) 0.709 ± 0.017 0.705 ±0.036 



show any sizeable CP violation in the SM, are examined. 
For these channels, the differential decay rate does not 
have a sin AmAt contribution, so that the coefficients 
analogous to sin 2(3 and cos 2(3 should vanish. No signif- 
icant deviation this expectation is observed (Table IX). 



TABLE IX: Fit results for the B ± 
samples. 



JfyK* ± data control 



Sample 


sin 2(3 


cos 2/3 


B ± J/tPiK^) 

B ± -> J/^A^TT ) 


0.21 ± 0.20 
0.20 ±0.20 


-0.21 ±0.47 
-0.26 ±0.46 



H. Systematic Uncertainties 

The contributions to the systematic uncertainty are 
summarized in Table X. Systematic uncertainties (a) - 
(j) and (q) - (t) are in common with the sm2(3 analysis 
[32] and are estimated in the same way. Systematic un- 
certainties (k) - (p) are specific to this J/0 (^s 71-0 ) anal- 
ysis and are elaborated in the following: 

(k) The systematic uncertainty due to imperfect knowl- 
edge of the fractions and CP values of the back- 
ground sources is obtained by varying the fractions 
(see Table VII) by one standard deviation, if the 
background is measured, or by 50% of the branch- 
ing fraction used in the Monte Carlo otherwise. 
The effective CP values (see Table VII) of unmea- 
sured background is set to —1 and then to ±1 to 
evaluate the effect on the measured parameters. 

(m) Backgrounds are assumed to have the same dilu- 
tions as the signal. To evaluate the related uncer- 
tainty, the dilutions obtained from the B^ sample 
are used and the difference in the results is taken 
as the systematic uncertainty. 

(n) Random sets of amplitude moduli and strong 
phases are generated, according to a multi- 
Gaussian distribution, based on the covariance ma- 
trix obtained in the fit for the amplitudes and 
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on the sytematic uncertainties in the amplitudes. 
These amplitudes are used in place of the nomi- 
nal amplitudes to evaluate the variation in the CP 
parameters. This procedure incorporates the un- 
certainties in the 5-wave amplitude as well. 

(o) The limited size of the Monte Carlo sample induces 
an uncertainty in the moments used to determine 
the acceptance and the background distribution. 
This is evaluated by splitting the Monte Carlo into 
ten samples, leading to ten CP measurements, and 
taking as an estimate of the uncertainty the RMS 
divided by vTO- 

(p) A flat angular distribution has been assumed for 
some of the background components (Sec. VI F). 
We estimate the magnitude of the related bias by 
computing the background moments from low-m,ES 
events. 



TABLE X: Summary of systematic and statistical uncertain- 
ties on sin 2/3 and cos 2/3. 



Source 


sin 23 


cos 2/3 


Signal Properties 


(a) Ai-resolution function 


±0.002 


±0.002 


(b) signal dilution Bcp vs Bfi av 


±0.012 


±0.013 


(c) Gaussian model for outliers 


±0.001 


±0.000 


(d) ftaii parameter 


±0.002 


±0.003 


(e) resolution/tagging correlation 


±0.001 


±0.001 


(f) SVT alignment 


±0.010 


±0.030 


Background properties: B 


flav 




(g) signal probability 


±0.001 


±0.001 


(h) ARGUS mo parameter 


±0.002 


±0.010 


(i) oscillating contribution 


±0.001 


±0.022 


(j) Speak contribution 


±0.001 


±0.003 


J/V> {K% tt u ) specific 


(k) background fraction and CP parity 


±0.032 


±0.142 


(m) background dilutions 


±0.002 


±0.006 


(n) amplitude uncertainties 


±0.016 


±0.154 


(o) statistics used for moments 


±0.030 


±0.030 


(p) angular background distribution 


±0.024 


±0.064 


External parameters 


(q) 2 scale and "6oosf 


±0.001 


±0.001 


(r) beam spot 


±0.010 


±0.040 


(s) 5° lifetime 


±0.014 


±0.040 


(t) Am 


±0.018 


±0.032 


Monte Carlo 


(u) Monte Carlo statistics 


±0.130 


±0.140 


Total systematic uncertainty 


±0.14 


±0.27 


Statistical uncertainty 


±0.57 


+0.76 

-0.96 



I. Results 

The results of the fit are given in Table XI. Figure 10 
shows the contour plots in the cos2/3, sin2/3 plane. We 



obtain 

cos2/3 = +3.32 tg;^(stat) ± 0.27(syst), 

sin2/3 = -0.10±0.57(stat)±0.14(syst). (60) 

The quality of the fit is estimated by generating 2000 ex- 
periments using the parametrized MC, and with the same 
sample size that is observed for the data. The probability 
to obtain a likelihood lower than that obtained from the 
data is found to be (22 ± 1)%. 

When sin2/3 is fixed to the value measured in B decays 
to J/ipK® and related modes, sin2/3 = sin2/3 = 0.731 [4], 
we find 

cos2/3 = ±2.72l£^(stat) ± 0.27(syst). (61) 

J. Graphical Representation 

The distribution of the time difference At is shown in 
Figs. 11(a) and 11(b), and the time-dependent asymme- 
try is shown in Fig. 11(c). Note that in the case of perfect 
acceptance this asymmetry is not sensitive to cos2/3 [27]. 

A graphical representation of the sensitivity of the data 
to cos2/3 is obtained from the time dependence of the 
moment of C. Since C is orthogonal to both A and S, we 
obtain, using Eq. (13) 

(C)±(At) = J g±(cj, At; A, sin 2/3, cos2/3)C(u>; A) du: 

= ±Eo e -r |At| x 
2 

jcos(AmAf) J V(u); A)C(u>; A) da; + 
sin(AmAi) cos 2/3 J C 2 {u>;A) du>j . 

(62) 

We see that the magnitude of the sin(AmAi) oscillation 
is proportional to cos 2/3. The introduction of the angu- 
lar acceptance e(w) in principle breaks the above orthog- 
onality, causing J ACedu and J SCedu) terms to appear 
in the (C)±(At) expression, in addition to the J VCedu) 
and J C 2 edui terms. These quantities are estimated using 
Monte Carlo and are found to be at the percent level of 
/ C 2 edu for / ACedu and / SCeduj, and 14% of / C 2 eduj 
for / "PCedu;. 

Figure 12 shows the moment of C as a function of At, 
overlaid with a function obtained from Eq. (62) that takes 
the acceptance into account. 

K. Confidence Level for Positive cos 2/3 Solution 

The value sin2/3 = 0.731 ± 0.056 [4] measured in the 
charmonium-AT channel is in good agreement with ex- 
pectations from the measurements of the sides of the Uni- 
tarity Triangle if the choice /3 « 0.41 = /3o is made. How- 
ever, the alternative solutions /3 w n/2 — 0.41, 0.41 + 7r, 
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TABLE XI: Global CP fit of the J/ip K*° (K° s it ) events together with the Bfi av sample. The transversity amplitudes used 
are those measured in the angular analysis. The cos2/3 value shown is the one corresponding to "Solution II" for the strong 
phases. The b's are the coefficients of the linear dependence of the At off-set on At uncertainty : (At) = b x a At- 



Parameter 


Value 


Correlation with sin 2/3 


Correlation with cos 2/3 


sin 2/3 


-0.10 ±0.57 


±1.000 


-0.368 


cos 2(3 


Q QQ + 0.76 

°- oz -0.96 


-0.368 


±1.000 




Sis 


;nal resolution function 




Score 


1.093 ± 0.048 


-0.020 


±0.028 


Stall 


3.0 (fixed) 






b core Lepton 


0.012 ±0.063 


±0.017 


-0.010 


bcore Kaon I 


-0.226 ±0.052 


±0.008 


-0.050 


bcore Kaon II 


-0.248 ± 0.046 


±0.013 


-0.023 


bcore Inclusive 


-0.212 ±0.047 


±0.020 


-0.020 


btail 


-1.01 ±0.29 


-0.021 


±0.029 


ftail 


0.109 ±0.020 


±0.022 


-0.030 


fout 


0.002 ± 0.001 


-0.004 


±0.006 


Signal dilutions 


(D), Lepton 


0.933 ±0.013 


-0.002 


-0.004 


(D), Kaon I 


0.799 ±0.014 


-0.009 


±0.051 


(D), Kaon II 


0.582 ±0.016 


-0.001 


-0.006 


(D), Inclusive 


0.368 ± 0.017 


±0.009 


±0.024 


AD, Lepton 


0.031 ±0.022 


-0.003 


-0.001 


AD, Kaon I 


0.023 ± 0.022 


-0.010 


±0.039 


AD, Kaon II 


0.090 ± 0.024 


-0.008 


±0.002 


AD, Inclusive 


0.050 ± 0.026 


±0.002 


±0.004 


Background properties (Bfi av ) 


T [ps] 


1.335 ±0.064 


±0.001 


±0.000 


/(r = 0) Lepton 


0.29 ±0.17 


±0.000 


-0.001 


/(t = 0) Kaon I 


0.631 ± 0.027 


±0.000 


-0.001 


/(r = 0) Kaon II 


0.659 ± 0.024 


±0.000 


-0.001 


/(r = 0) Inclusive 


0.684 ± 0.023 


±0.000 


±0.000 


Background resolution function 


Score 


1.398 ± 0.019 


±0.001 


-0.002 


bcore 


-0.043 ±0.013 


-0.001 


±0.002 


fout 


0.015 ± 0.002 


±0.000 


±0.001 


Background dilutions 


(D), Lepton , r = 


1.36 ±0.69 


±0.000 


±0.001 


(D), Kaon I , r = 


0.648 ± 0.030 


±0.001 


-0.004 


(D), Kaon II , r = 


0.393 ± 0.023 


±0.000 


±0.000 


(D), Inclusive , r = 


0.158 ±0.024 


-0.001 


-0.002 


(D), Lepton , r > 


0.17 ±0.11 


±0.000 


±0.000 


(D), Kaon I , r > 


0.251 ±0.048 


±0.000 


±0.000 


(D), Kaon II , r > 


0.278 ± 0.042 


±0.000 


±0.000 


(D), Inclusive , r > 


0.031 ± 0.046 


±0.000 


±0.000 



and 37r/2 — 0.41 could turn out to be correct if there 
is a significant contribution from outside the Standard 
Model. We show here that we can exclude at a sig- 
nificant level of confidence the possibilities tt/2 — 0.41 
and 3tt/2 — 0.41, assuming that the value of sin2/3 that 
would be inferred from a high-statistics measurement of 
the J/ipK* channel would conform to the measurement 
of Ref. [4]. We therefore constrain sin2/3 to sin2/?o- The 
systematic uncertainty on /?, induced by the uncertainty 
in sin2/3, (±0.056) [4] is ±0.043, which is negligible here. 

We define cos2/3 = - sin 2 2/3 w ±0.68. In the 

following, we estimate the confidence level at which the 



— cos2/3 hypothesis can be excluded against the ± cos2/3 
solution. 



1. Assuming Gaussian Statistics 

Figure 13 shows the variation of the likelihood as a 
function of cos2/3. In the case of fixed sin2/3, the opti- 
mum is obtained at cos2/3 = ±2.72, 2.2<r from ±cos2/3 
and 3.5cx from — cos2/3 - For a Gaussian distribution, 
the probabilities to observe values 2.2 and 3.5 a from 
the mean value are, respectively, 3.25% and 0.08%. In 
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-1 1 2 3 4 5 

cos2p 



FIG. 10: Contour plots in the cos2/3, sin2/9 plane. The triangle denotes the result of the fit. The error bars show the statistical 
uncertainty and the quadratic sum of the statistical and systematic uncertainties. The star denotes the result of the fit with 
sin2/3 fixed at sin2/3 = 0.731 [4]. The value of sin2/3 of reference [4] and its uncertainties are represented as dashed horizontal 
lines. The na(n = 1, 2, 3) contour corresponds to a decrease of 0.5n 2 in the log-likelihood with respect to the maximum value. 
The unit circle (cos 2 2/3 4- sin 2 2/3 = 1) on which the true values must lie is also shown. 



a Bayesian approach, assuming equal a priori probabili- 
ties for the ± cos2/?o hypotheses, the probability that the 
+ cos2/3 choice is wrong would be 0.08/(3.25 + 0.08) = 
2.4%. 



2. Using the Distribution of cos2/3 Values Obtained from 
Simulated Experiments 

To take into account the nonparabolic shape of the 
log-likelihood as a function of cos2/3, cos2/3 values are 
measured with 2000 simulated samples, each the same 
size as the data sample (104 events) (Fig. 14). For the 
+ cos2/3 hypothesis, the distribution d d c ^ s2 p of cos2/3 val- 
ues is that shown in Fig. 14(c). An unbinned likelihood 
fit is performed to the sample of the 2000 cos2/3 values, 
with a sum of two Gaussian functions, /i(cos2/3). The fit 
result is shown in Fig. 15(a) (where /i(cos2/3) is scaled by 
100, i.e., 2000 times the bin size). 

The distribution Ji^ a is obtained by the transforma- 



tion cos2/3 



d cos2/3 

cos2/3: i.e., we have 



dN+ 
g?cos2/3 



(cos2/3) 



dN~ 
dcos2f3 



(-cos2/3). 



In a frequentist approach, we consider the probabil- 
ity that a result would disfavor, by at least as much 
as ours, the — cos2/3 hypothesis against the + cos2/3 



one, by computing the probability to observe a ratio 
r(cos2/3) = 3 ^(cos2/3)/ a ^(cos2/3) smaller than or 
equal to the one we obtain, r(+2.72) [42]. Because this 
ratio r(cos2/3) has a monotonic decreasing behaviour with 
cos2/3, the probability to obtain r(cos2/3) < r(+2.72), if 
the true cos2/3 value is indeed — cos2/3q, is 



dN- 



+2 .72 C?COs2/3 

0.6%, 



(cos2/3) dcos2/3 



(63) 



leading to the confidence level at which the — cos2/3o hy- 
pothesis is excluded: 

CL Froq - (- cos 2/3 excluded) = 1-a 

= 99.4%. (64) 

If we ask how likely it is to obtain a result in the above 
(+2.72, +oo) range, if the true value of cos2/3 is +cos2/?o, 
we find: 



dN^ 



2.72 dcos2(3 



(cos2/3) dcos2/3 = 5.7%. (65) 



In a frequentist interpretation, a high value for this 
last quantity would have indicated, together with the 
high CL Frcq (— cos2/3 excluded) value obtained, that 
the (+2.72, +oo) domain would have allowed a sharp 
distinction between the two ±cos2/3 hypotheses. The 
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8 10 

At (ps) 



FIG. 11: The distribution of At for events in the signal region, 
for (a) B° and (b) B° tags with the fit result (full curve) over- 
laid. In (c) we show the raw asymmetry in the number of B° 
and B° tags in the signal region, (iV B o — Ng )/(N B o +Ng ), 
for data, with the fit result (full curve) overlaid. Note that 
above distributions are not sensitive to cos2/3 since this de- 
pendence vanishes when integrated over the angular variables. 



0.02 - 



0.01 



-0.01 - 




-0.02 



FIG. 12: The moment of C as a function of At. The overlaid 
curve corresponds to the fit results (Table XI). The dashed 
curve corresponds to cos2/3 = +yl — sin 2 2/3q = +0.68 [4], 



the dotted one to the non-standard solution cos2/3 : 



-0.68. 



(J 
e 




FIG. 13: The negative logarithm of the likelihood as a func- 
tion of cos2/3. Continuous line: sin2/3 is a free parameter. 
Dashed line: sin2/3 is fixed at sin2/3o = 0.731 [4]. 



rather low value observed here (5.7%) expresses that, at 
the present level of statistics, the discrimination between 
the ±cos2/3o hypotheses is rather modest. We can 
conclude however that our result would be somewhat 
more improbable (0.6%) if the true value of cos2/3 were 
— cos2/3 than it would be (5.7%) if the true value were 
+ cos2/3o- 

In a Bayesian approach, assuming that the two 
±cos2/?o hypotheses have a priori equal probabilities, 
the confidence level at which the — cos2/3o solution 
is excluded, CL(— cos2/3o excluded), is obtained from 

CT(+ 2 - 72 ) and ct(+ 272 ) as follows: 

h(+2 72) 

CL(- cos 2/? excluded) = fe(+2J2) + fe( _ 2-72) 

6.64 ±0.38 
(6.64 ± 0.38) + (0.86 ±0.15) 
= (88.6 ±2.0)%. (66) 

The probability to select incorrectly the + cos2/3o solu- 
tion is significantly larger than for the previous Bayesian 
estimate based on Gaussian statistics (Sec. VIK1). 



The uncertainty in Eq. (66) comes from the sta- 
tistical uncertainties on /i(+2.72), h(— 2.72) (limited 
by the 2000 simulated experiments used), and their 
correlation (—6%). The systematic effects on the 
cos 2(3 measurement contribute to a ±0.4% variation of 
CL{— cos2/?o excluded) and are included in quadrature 
in the above uncertainty. We include a — ler safety mar- 
gin on CL(— cos2/?o excluded), and thus report 

CL{- cos 2/3 excluded) = 86.6%. (67) 



VII. CONCLUSION 

We measure the transversity amplitudes of the de- 
cay to flavor eigenstates, B — > J/4>K*°(K ± tt t ) and 
B — > J/ijjK^ (K ± ir° and X°7r ± ), with improved pre- 
cision with respect to existing measurements. We deter- 
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5 
Fitted cos2|3 value 



FIG. 14: The distribution of the values of cos2/3 ((a) and (c)), 
and of the statistical uncertainties ((b) and (d)), obtained 
in 2000 simulated experiments, each based on a sample of 
the same size as the data; i.e., 104 events. These are taken 
from the parametrized MC sample mentioned above, with the 
generated cos2/3 value +0.68. In (a) and (b) sin2/3 is also free 
in the fit. In (c) and (d), sin2/3 is fixed to the world average. 
The vertical arrows show the positions of the values obtained 
from the data. 



mine 



<5|l - S 
S± - S 
\A \ 2 
l^lll 2 
l^l 2 



(-2.73 ±0.10 ±0.05) rad, 
(+2.96 ±0.07 ±0.05) rad, 
0.566 ±0.012 ± 0.005, 
0.204 ±0.015 ± 0.005, 
0.230 ±0.015 ±0.004, 



and 



6n-6 ± = (0.60 ±0.08 ±0.02) rad. 



(68) 



(69) 



We observe the presence of a significant S-w&ve ampli- 
tude interfering with the P-wave amplitude in the region 
of the if * (892). Using a novel method based on the de- 
pendence on the Kit invariant mass of the interference 
between the S- and P- waves, we resolve the ambiguity 
in the determination of the strong phases involved in B 
decays to J/ipK *(892). 

The values obtained for |A|| | 2 and \ A±\ 2 are consistent 
with being equal. The additional unambiguous determi- 
nation of the phases relative to that of Aq indicates that 
they have similar size but opposite sign, with a difference, 
5|| — S±, of 34 ± 5 degrees. Using the relations between 
the helicity amplitudes and the transversity amplitudes, 



H +1 = 



H a = 



{A\\+A ± )/y/2 

\H-\e iS ~, 
A , 




-3.25 -3 -2.75 -2.5 -2.25 

cos2|5 



1 1 1 1 1 1 ii i 

2.25 2.5 2.75 3 3.25 
cos2|5 



FIG. 15: (a) The cos2/3 distribution for 2000 simulated ex- 
periments of the same size as the data sample, where the 
generated values for sin2/3 and cos2/3 are +0.731 and +0.68, 
respectively (same as Fig. 14(c)). An unbinned likelihood fit is 
performed to this distribution, using the sum of two Gaussian 
functions. The fit result is shown on the plot (full line) with 
the individual Gaussian contributions (dashed lines). The 
long vertical arrows show the cos2/3 values ±2.72. The small 
arrows indicate the extent of systematic uncertainties, (b) 
and (c) zoom on the cos2/3 = ±2.72 regions of Fig. 15(a). The 
densities of points at +2.72 and —2.72 are used to discriminate 
between the cos2/3 = ±\/l — sin 2 2/3o = ±0.68 hypotheses, as 
explained in the text. 



TABLE XII: Helicity-amplitude moduli and phases for H+ 
and H- obtained from the measured transversity amplitudes 
(Eq. (68)) using Eq. (70). The corresponding configuration is 
shown in Fig. 16. The uncertainties are statistical only. 



\H + \ 2 


\H-\ 2 


S+ ( rad) 


S- (rad) 


0.396 + 0.015 


0.0379 ± 0.009 


-3.04 ± 0.08 


-1.36 + 0.12 



(70) 



we obtain the moduli and phases given in Table XII. This 
determines the hierarchy of the helicity amplitudes in the 
decay to be \H \ : \H +1 \ : - 0.75 : 0.63 : 0.19. The 

corresponding configurations of the helicity and transver- 
sity amplitudes in the complex plane are illustrated in 
Fig. 16. 

We confirm the presence of nonzero relative strong 
phases, with the difference between the phases of A and 
A 1 1 deviating from tt with a significance of 3.6er, and the 
phase difference between the two transverse amplitudes 
being 7.6ct from zero. 

Treating sin 2/3 and cos 2/3 as independent quan- 
tities in the fit to the data, we obtain cos 2/3 = 
+3.32 to:9e(( stat )) ± 0.27((syst)). When sin2/3 is fixed 
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FIG. 16: Representation in the complex plane of the mea- 
sured transversity amplitudes (Ao, Au , A±) (Eq. (68)) and the 
equivalent helicity amplitudes (Ho, H-i, -H+i) obtained using 
Eq. (70). The values for H+ and H- are quoted in Table XII. 



to the value measured in the charmonium-_ftT° modes, 
sin2/3 = 0.731 [4], we find 

cos2/3 = +2.72±°^° ± 0.27. (71) 

The sign of cos 2/3 is found to be positive at the 86% 
C.L. This is compatible with the sign inferred from the 
Standard-Model-based fits of the Cabibbo-Kobayashi- 
Maskawa triangle, thus limiting the possible presence of 
unknown physics beyond the Standard Model. 
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of our PEP-II colleagues in achieving the excellent 
luminosity and machine conditions that have made 
this work possible. The success of this project also 
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where 

• Cov H [A] is the covariance matrix of A at the max- 
imum of L', estimated by the HESSE routine of 
Minuit [38] after the fit has converged. 

• In the expressions for the bilinear forms 
/ b(u>)(Vg b ' obs (u;)/g b ' obs (u;)) 2 duj and 

(jb(u>)Vg b > obs (u>)/g b > obs (u:)dLO S j , V denotes the 

gradient, i.e., differentiation with respect to the 
fit parameters A. The "square" is not to be 
understood as a "scalar product" , but as a "direct 
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APPENDIX A: UNCERTAINTIES WITH A 
PSEUDO-LOG-LIKELIHOOD, AND 
VALIDATIONS 



In the angular analysis, the background correction is 
performed using a pseudo-log- likelihood L' , defined in 
Eq. (26). As V is not a log- likelihood, the uncertainties 
yielded by the minimization program (Minuit) [38] that 
is used are incorrect estimates of the actual uncertainties. 
The correct estimate is given by 



(Al) 



product", i.e. v 2 = v^v, so that the resulting 
quantity is a square matrix. 



• g b ' obs (u)) and b(u>) are the PDFs for the signal and 
the background. Note that in practice the knowl- 
edge of the PDF of the background is not needed 
for the computation of Eq. (Al) because for any 
function h(u>), J b(u>)h(u>)du) is estimated by the 
average of h over the hzes sideband background 



Cov[A] = Cov H [A]{ 1 + 



n B (l + k) / 6(w) 



Vg b ' obs (u;) 
(wT 



-.b.obs 



du> 



b(u}) — rTT — dw 

gb,obs^ 



Cov H [A] , 
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sample: 



N B 

jb{ U )h( U )dw«^-^h( Ui ). (A2) 



• k is the scaling parameter k 
its uncertainty. 



ns/Ns, and a k is 



The estimated number of background events in the sig- 
nal region ns is obtained from an ARGUS plus Gaussian 
fit to the toes spectrum. 

The validation of the pseudo-log-likelihood method 
(i.e., the unbiased nature of the fit parameters, which 
is not shown here, and of their uncertainties) comes 
from MC-based studies. We have simulated 10 3 experi- 
ments with 10 4 events each [35], using a signal PDF with 
9a = 4>A = S\\ = S± = 1 rad (as defined in Eq. (5)). 

We study the behavior of the fit for various values 
of the purity, adding the appropriate number of back- 
ground events. A variety of background shapes have 
been used. Figure 17 presents results using an AR- 
GUS Toes distribution with an angular distribution of 
a + bcos 2 (9 K «) +ccos 4 (^») [43]. 
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FIG. 17: Root mean square (RMS) of the pull distributions 
as a function of signal purity (denned in the 5.2 - 5.3 GeV/c 2 
jtibs range), for the fitted parameters $A,(j>A,S\\, and S±. 
Open symbols denote the RMS of the pulls computed with 
the uncertainties taken directly from Minuit. Closed symbols 
denote the uncertainties computed according to Eq. Al. 



Figure 17 shows the results from the Monte Carlo 
study. As the purity decreases, the Minuit-reported un- 
certainties diverge more and more from the actual spread 
in the results. The uncertainties calculated from Eq. (Al) 
correctly predict the behavior of the spread, even at low 
purity. 
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